Skip to main content
Skip Navigation

NAEP Technical DocumentationSchool Sample Selection for the 2013 State Assessment

         

Computation of Measures of Size

School Sample Sizes: Frame and New School

Evaluation of the Samples Using State Achievement Data  

For the grades 4, 8, and 12 public school state assessment samples, schools were sampled independently from each jurisdiction with probability proportional-to-size (PPS) using systematic sampling. Prior to sampling, schools in each jurisdiction were sorted by the appropriate implicit stratification variables (urbanicity status, race/ethnicity status, and achievement score or zip code-based median household income) in a serpentine order. A school's measure of size was a complex function of the school's estimated grade enrollment. Schools whose measure of size was larger than the sampling interval could be selected or “hit” multiple times. Schools with multiple hits were selected with certainty and had larger student sample sizes.

The sampled schools for the public school state assessment samples came from two frames: the public school sample frame (as constructed from the  Common Core of Data (CCD)) and the new-school sampling frame.

Schools from the CCD-based frame were sampled at a rate that would yield specific target student sample sizes for each jurisdiction. At grades 4 and 8, jurisdictions had a target sample size of 6,600 students - 3,150 students each for the reading and mathematics operational assessments and 300 students for pilot tests. For the special mathematics assessment in Puerto Rico, the target sample size was 5,800 students. By design, Bureau of Indian Education (BIE) schools were not part of the state assessments this year. However, separate BIE school samples were selected based on target student sample sizes that were large enough to ensure that BIE schools were sufficiently represented in the national samples.

At grade 12, the target sample sizes of students differed by jurisdiction and are shown in the following table. These numbers reflect the desired number of assessed students for the reading and mathematics operational assessments (2,300 students per subject) and an upward adjustment to offset expected rates of school and student attrition due to nonresponse and ineligibility.

Target sample sizes of assessed students, grade 12 state assessment, by jurisdiction: 2013
JurisdictionTarget student sample size
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.
Arkansas6,200
Connecticut6,750
Florida6,600
Idaho6,250
Illinois7,250
Iowa6,850
Massachusetts6,250
Michigan7,400
New Hampshire8,350
New Jersey6,500
South Dakota6,500
Tennessee7,400
West Virginia6,650

Prior to selection, schools were deeply stratified in each jurisdiction to ensure that the school sample distribution reflected the school population distribution as closely as possible, with regard to the stratification variables, to miminimize sampling error. The success of this approach was shown by comparing the proportion of minorities enrolled in schools (based on CCD values for each school), median income, and urban-centric locale (viewed as an interval variable) reported in the original frame against the school sample.

In addition, the distribution of state assessment achievement scores for the original frame can be compared with that of the school sample for those jurisdictions for which state assessment achievement data are available, as was done in the evaluation of the samples using state achievement data. The number of significant differences found in this analysis was smaller than what would be expected to occur by chance, given the large number of comparisons that were made. The number of significant differences remained small even with the use of a finite population correction factor in the calculation of the sampling variances. The close adherence of sample values to frame values suggested there is little evidence that the school sample for NAEP 2013 is not representative of the frame from which it was selected. The achievement/median income variable is used as the third-level sort order variable in the school systematic selection procedure. While it may be a rather low-level sort variable, it still helps control how representative the sampled schools are in terms of achievement. The close agreement between frame and sample values of these achievement/median income variables provided assurance that the selected sample is representative of the frame with respect to achievement status.


Last updated 17 April 2017 (GF)