NAEP Technical DocumentationSampling of Public Schools for the 2018 Social Sciences Assessment

In the design of each school sample, six objectives underlie the process of determining the probability of selection for each school and the number of students to be sampled from each selected school containing grade-eligible students. The six objectives are

• to meet the overall target student sample size;
• to select an equal-probability sample of students;
• to limit the number of students selected from any one school;
• to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included;
• to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools; and
• to increase the number of American Indian/Alaska Native (AIAN), Black, and Hispanic students in the sample.

The goal in determining the school's measure of size is to optimize across the middle four objectives in terms of maintaining the accuracy of estimates and the cost-effectiveness of the sample design.

Therefore, to meet the target student sample size objective and achieve a reasonable compromise among the next four objectives, the following algorithm was used to assign a measure of size to each school based on its estimated grade enrollment as indicated on the sampling frame.

The measures of size vary by enrollment size. The initial measures of size (MOS) were set as follows:

A school with more than 5 percent AIAN students and at least 5 AIAN students in the sample grade is in the high AIAN stratum for NAEP. The measures of size for schools in the high AIAN stratum are quadrupled to increase their chances of selection. A school that is not in the high AIAN stratum and with more than 15 percent Black and Hispanic students and at least 10 Black or Hispanic students in the sample grade is in the high Black/Hispanic stratum for NAEP. The measures of size for schools in the high Black/Hispanic stratum or in the Honolulu primary sampling unit (PSU) are doubled to increase their chances of selection:

Schools in the Honolulu PSU have their measures of size doubled to ensure at least one sampled school from the PSU. The Honolulu PSU is a certainty not due to its size, but because it is unique.

The next task in this development is to describe b_j, the constant of proportionality for each grade. It is a sampling parameter that, when multiplied with a school’s preliminary measure of size (M_js), yields the school’s final measure of size. It is computed in such a way that, when used with the systematic sampling procedure, the target student sample size is achieved. For social sciences public schools, b_j is 0.000112622 for eighth grade.

The final measure of size is defined as:

The quantity u_j (the maximum number of “hits” allowed) in this formula is designed to put an upper bound on the burden for the sampled schools. For public schools, u_j is 1 because by design a school could not be selected, or "hit," in the sampling process more than once within a grade.

In addition, new and newly-eligible schools were sampled from the new school frame. The final measure of size for these schools is defined as:

The variable π_djs is the probability of selection of the district d into the new-school district sample.

Schools were ordered within each jurisdiction using the serpentine sort described under the stratification of public schools. A systematic sample was then drawn using this serpentine-sorted list and the measures of size. The number of public schools selected for social sciences was approximately 800, including approximately 10 new or newly-eligible schools.

Printer-friendly Version