​​​​NAEP Technical DocumentationComparison of Multiple Groups

			Bonferroni Procedure False Discovery Rate Procedure

To ensure that significant differences in NAEP data reflect actual differences and not mere chance, error rates need to be controlled when making multiple simultaneous comparisons. The more comparisons that are made (e.g., comparing the performance of White, Black, Hispanic, Asian/Pacific Islander, and American Indian/Alaska Native students), the higher the probability of finding significant differences by chance. In NAEP, the Benjamini-Hochberg False Discovery Rate (FDR) procedure is used to control the expected proportion of falsely rejected hypotheses relative to the number of comparisons that are conducted. NAEP employs a number of rules to determine the number of comparisons conducted, which in most cases is simply the number of possible statistical tests. However, when comparing multiple years, the number of years does not count toward the number of comparisons.

There are several ways to take into account how many related comparisons are being made. NAEP used only the Bonferroni procedure prior to the 1996 assessment. Starting with the 1998 assessment, NAEP has used only the Benjamini-Hochberg False Discovery Rate procedure.

Printer-friendly Version