NAEP Technical DocumentationComputation of Measures of Size for the 2017 Private School National Writing Assessment

There were six objectives underlying the process for determining the probability of selection for each school and for setting the number of students to be sampled within each selected school:

  • to meet the target student sample size for each grade;
  • to select an equal-probability sample of students;
  • to limit the number of students selected from any one school;
  • to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included;
  • to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools; and
  • to increase the number of Black and Hispanic students in the sample.

The goal in determining the school's measure of size is to optimize across the middle four objectives in terms of maintaining the precision of estimates and the cost effectiveness of the sample design.

To increase the number of Black and Hispanic students in the sample, the measures of size for schools with relatively high proportions of Black and Hispanic students (15 percent or more and with at least 10 Black or Hispanic students in grades 4 and 8) were doubled.

In the formulas below, xjs is the estimated grade enrollment for private school type j and school s, yj is the target within-school student sample size for stratum j, and zjs is the within-school take-all student cutoff for stratum j to which school s belongs, and Ps is a primary sampling unit (PSU) weight associated with the private school universe (PSS) area sample.

For grades 4 and 8, the target sample sizes and take-all cutoffs were both 50.

For schools with high proportions of Black and Hispanic students, the preliminary measures of size (MOS) were calculated as follows:

MOS subscript js equals 2 times P subscript s times bracket matrix 4 rows 2 columns. Column 1 equals x subscript js, y subscript j, open paren y subscript j divided by 20 close paren times x subscript js, y subscript j divided by 4. Column 2 equals if z subscript js less than x subscript js, if 20 less than x subscript js less than or equal to z subscript js, if 5 less than x subscript js less than or equal to 20, x subscript js less than or equal to 5  

For all other schools (those with a low proportion of Black and Hispanic students), the preliminary measures of size (MOS) were calculated as follows:

MOS subscript js equals P subscript s times bracket matrix 4 rows 2 columns. Column 1 equals x subscript js, y subscript j, open paren y subscript j divided by 20 close paren times x subscript js, y subscript j divided by 4. Column 2 equals if z subscript js less than x subscript js, if 20 less than x subscript js less than or equal to z subscript js, if 5 less than x subscript js less than or equal to 20, x subscript js less than or equal to 5 

The preliminary school measure of size was rescaled to create an expected number of hits by applying a multiplicative constant bj, which varies by grade and school type. For the 2017 private school samples, the limit was one hit. 

It follows that the final measure of size, Ejs, was defined as:

E subscript js equals the minimum of open paren b subscript j times MOS subscript js comma u subscript j close paren      

where uj  is the maximum number of hits allowed.

The school's probability of selection, PI subscript js , was given by:

PI subscript js equals the minimum of open paren E subscript js comma 1 close paren     

One can choose a value of bj such that the expected overall student sample yield matches the desired targets specified by the design, where the expected yield is calculated by summing the product of an individual school’s probability and its student sample yield across all schools in the frame.

In addition, new and newly eligible Catholic schools were sampled from the new-school frame. The assigned measures of size for these schools,

E subscript js equals the minimum of open paren b subscript j times MOS subscript js times the inverse of pi subscript djs comma u subscript j close paren,  

used the bj and uj values from the main school sample for the grade and school type (i.e., the same sampling rates as for the main school sample). The variable πdjs is the probability of selection of the diocese into the new-school diocese (d) sample.

In addition, an adjustment was made to the initial measures of size in an attempt to reduce school burden by minimizing the number of schools selected for both the national mathematics and reading assessments and the national writing assessments in private schools. The NAEP sampling procedures used an adaptation of the Keyfitz process to compute conditional measures of size that, by design, minimized the overlap of schools selected for both the mathematics/reading and writing assessments.

Last updated 29 August 2022 (SK)