​​​​​​​​​NAEP Technical DocumentationComputation of Measures of Size for the 2022 Eighth-Grade Private School National Assessment in Civics and U.S. History

In designing the eighth-grade private school civics and U.S. history assessment samples, five objectives underlie the process of determining the probability of selection for each school and the number of students to be sampled from each selected school:

• to meet the target student sample size for each grade;
• to select an equal-probability sample of students;
• to limit the number of students selected from any one school;
• to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included; and
• to reduce the sampling rate of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools.

The goal in determining the school's measure of size is to optimize across the last four objectives in terms of maintaining the precision of estimates and the cost effectiveness of the sample design.

Therefore, to meet the target student sample size objective and achieve a reasonable compromise among the next other objectives, the following algorithm was used to assign a measure of size to each school based on its estimated grade enrollment as indicated on the sampling frame.

In the formula below, `x_{js}` is the estimated grade enrollment for stratum `j` and school `s`, `y_{j}` is the target within-school student sample size for stratum `j`, and `z_{js}` is the within-school take-all student cutoff for stratum `j` to which school `s` belongs, and `P_{s}` is a primary sampling unit (PSU) weight associated with the private school universe (PSS) area sample.

For grade 8, the within-school target sample size (`y_{j}`) was 50 and take-all cutoff was 52.

The preliminary measure of size (MOS) was calculated as follows:

\begin{equation}
MOS_{js} =
P_{s} \times \left\{ \begin{array}{l}
x_{js} & \text{if } z_{j} < x_{js} \\[2pt]
y_{j} & \text{if } 20 < x_{js} \leq{z_{j}} \\[2pt]
\left(\dfrac{y_j}{20}\right) \times x_{js} & \text{if } 5 < x_{js} \leq {20} \\
\dfrac{y_j}{4} & x_{js} \leq {5} \end{array}\right.
\end{equation}

The preliminary school measure of size was rescaled to create an expected number of hits by applying a multiplicative constant `b_{j}`, which varies by school type. 

It follows that the final measure of size, `E_{js}`, was defined as: \begin{equation}
E_{js}=min(b_{j}\times MOS_{js}, u_{j}),
\end{equation} where `u_{j}` is the maximum number of hits allowed. For the 2022 private schools sample, the limit was one hit. 

One can choose a value of `b_{j}` such that the expected overall student sample yield matches the desired targets specified by the design, where the expected yield is calculated by summing the product of an individual school’s probability and its student sample yield across all schools in the frame.

The school's probability of selection `pi_{js}` was given by: \begin{equation}
\pi_{js}=min(E_{js},1).
\end{equation}

In addition, new and newly-eligible Catholic schools were sampled from the new-school frame. The assigned measures of size for these schools, \begin{equation}
E_{js}=min(b_{j}\times MOS_{js}\times \pi_{djs}^{-1} , u_{j}),
\end{equation} used the `b_{j}` and `u_{j}` values from the main school sample for the grade and school type (i.e., the same sampling rates as for the main school sample). The variable `pi_{djs}` is the probability of selection of the diocese into the new-school diocese `d` sample.

Printer-friendly Version