NAEP Technical DocumentationComputation of Measures of Size for the 2017 State Assessment

In designing each school sample, five objectives underlie the process of determining the probability of selection for each school and how many students are to be sampled from each selected school containing the respective grade:

• to meet the target student sample size for each grade;
• to select an equal-probability sample of students;
• to limit the number of students selected from any one school;
• to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included; and
• to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools.

The goal in determining the school's measure of size is to optimize across the last four objectives in terms of maintaining the accuracy of estimates and the cost effectiveness of the sample design. In certain jurisdictions, a census of students was taken so as to meet, as nearly as possible, the target student sample size. Elsewhere, to meet the target student sample and achieve a reasonable compromise among the other four objectives above, the following algorithm was used to assign a measure of size to each school based on its enrollment per grade as indicated on the sampling frame. 

The preliminary measures of size (MOS_js) were set as follows:

where xjs is the estimated grade enrollment for school s in jurisdiction j, yj the target within-school student sample size for jurisdiction j, and zjs the within-school take-all student cutoff for jurisdiction j to which school s belongs

For the state samples at grades 4 and 8, the target sample sizes and take-all cutoffs were 62 and 75 for all jurisdictions with the exception of Puerto Rico, where they were 50 and 55, respectively. For large TUDAs (New York City, Los Angeles, Chicago, Miami-Dade, Clark Co., and Houston), the target sample sizes were 66 and take-all cutoffs were 75. For the remaining TUDAs, the target sample sizes were 74 and take-all cutoffs were 80.

The preliminary measure of size reflects the need to lower the expected number of very small schools in the sample, as the marginal cost for each assessed student in these schools is higher. These very small schools are sampled at half the rate of the larger schools, and their weights are doubled to account for the half sampling.

The next task in this development is to describe b_j, the constant of proportionality for a specified jurisdiction. It is a sampling parameter that, when multiplied by a school’s preliminary measure of size (MOS_js), yields the school’s final measure of size. It is computed in such a way that, when used with the systematic sampling procedure, the target student sample size is achieved.

The final measure of size, E_js, is defined as:

The quantity u_j (the maximum number of “hits” allowed) in this formula is designed to put an upper bound on the burden for the sampled schools. In most jurisdictions, u_j was set to 3. In Alaska, u_j was set to 8, and in Puerto Rico, u_j was set to 1.

In addition, new and newly-eligible schools were sampled from the new-school frame. The assigned measures of size for these schools,

,

Printer-friendly Version