- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Technical DocumentationComparisons of the State NAEP School Samples Against the CCD Frame
The full school sample is a probability sample from the frame for each jurisdiction. The summation of the probability-weighted estimated grade enrollment aggregations (estimated grade enrollment divided by the school probability of selection) should provide unbiased estimates of the corresponding frame grade enrollment figures.
This page summarizes differences between the weighted sample estimates and frame values for selected characteristics for fourth and eighth grades. Comparisons occurred among such sociodemographic variables as percentages of selected minorities (Black, Hispanic, Asian, and American Indian students), median income, type of location, and achievement scores. Two-sided p-values were calculated to test the null hypothesis that the difference between sample and frame is zero, using the jackknife standard error of the sample aggregation (note the assumption that the frame aggregation has no sampling error). It should be expected that many of the p-values would be small simply from randomness, as so many p-values were calculated.
The results are summarized as follows:
-
Of the differences that are significant at the 5 percent level, the absolute differences involving percentages are only a fraction of a percentage point, with most being near zero.
-
The largest significant difference in median income, again at the 5 percent level, represents less than a 5 percent relative difference between the two data sources.
-
There were no significant differences in achievement scores between the two sources at either grade.
-
Of 400 total differences that were calculated for fourth grade, only 14 reached the nominal 5 percent level of significance, fewer than the number that would be expected by chance.
-
Of 338 total differences that were calculated for eighth grade, only eight reached the nominal 5 percent level of significance, again fewer than the number that would be expected by chance.
Last updated 09 June 2008 (DB)
