Table of Contents  |  Search Technical Documentation  |  References

NAEP Sample Design → NAEP 2002 Sample Design → 2002 National Main Assessment Sample Design → School Sample Selection for the 2002 National Main Assessment → School Selection for Twelfth-Grade Public Schools

NAEP Technical DocumentationSchool Selection for Twelfth-Grade Public Schools

			Targets for Twelfth-Grade Public School Sample Frame and Sample Sizes by Minority Strata, Twelfth-Grade Public School Sample Overlap Control of Twelfth-Grade School Sample With the Education Longitudinal Study

For twelfth-grade schools, the overall student target for public schools was 26,400 for the operational assessments, with 39.3 percent assigned to the reading assessment and 60.7 percent to the writing assessment. In setting the school sample target, it also allowed for the link sample to NAEP 1998, the target of which was 11,700. The school sample was based on a total target of 38,100. Oversampling of minority students (Black and Hispanic) was also carried out by assigning doubled measures of size (MOS) to schools with 15 percent or more Black and Hispanic students and more than 10 Black and Hispanic students all together in the twelfth grade (called the "high-minority stratum").

As with the state assessment school sample selection, the general goal is to achieve a self-weighting sample at the student level. In this case, the target sample size within each school was 136 (rather than 60 as in fourth and eighth grade). As with fourth and eighth grade, schools with very small twelfth-grade enrollments were sampled at a lower rate. In this case, schools with five or fewer twelfth-graders were sampled at a 1/4 relative rate (the students in these schools have a probability of selection 1/4 that of schools with 20 or more students). Define x_s as the twelfth-grade enrollment for the school (derived from the CCD record for the school). The initial measure of size MOS_s for each school s on the twelfth-grade public school frame was computed as follows from x_s (each of the four pairs following the equal sign correspond to a condition on x_s and a functional relationship between MOS_s and x_s which is used to assign MOS_s under that condition. For example, schools with enrollment x_s greater than or equal to 150 had their MOS_s set equal to x_s) :

Note the "sliding scale" for schools with 6 through 19 enrollment. The relative rate increases as a linear function of enrollment from a relative rate of 1/4 for schools with 5 students to a relative rate of 1 for schools with 20 students. Schools with 20 or more twelfth-graders all have the same overall student sampling rate.¹

The probability of selection for each school is essentially this MOS_s multiplied by a constant of proportionality b, which is computed to achieve the target of T equal to 38,100 as closely as possible.

The final yield y_s (equal to the assessment sample size) for each sampled school was computed as follows, using the "almost all" function from the state assessment school sample selection, with new parameters (note that the two pairs within the parentheses correspond to two conditions on enrollment x_s [enrollment less than 150 or not], with two differing assignments of y_s as a function of x_s for each of these two conditions: y_s is equal to enrollment if enrollment is less than 150, and is equal to 136 otherwise):

All students were taken for schools up to 149 enrolled students, and 136 students were sampled in schools with 150 or more enrolled students. The sample design can be defined in terms of each school’s probability of selection, as follows (the formula below defines in effect two formulas for the probability of selection conditional on whether or not the school is in the high minority stratum, or not):

where H is the high-minority stratum. The quantity b is computed in an iterative process. It starts with a guess for b: b(1). The k-th iteration computes an overall expected yield of students as follows:

with S being the frame set of schools and E_k indicating expectations using b(k). The value of T(k) is compared with the desired target T of 72,000. If T(k) does not equal T (rounded to an integer), then it continues to a k+1st iteration, computing b(k+1) as some value in the interval [b,b(k)*T/T(k)], and continues iteratively until a b(K) is found that satisfies

rounded to the nearest integer. The final b value was 0.000127291. The final desired expected measures of size for each school were the E_K values computed using this final b. These are denoted as 'E_s' in Overlap Control of Twelfth-Grade School Sample with the Education Longitudinal Study. When this expectation was less than 1, then it is the probability of selection π_s for the school. When E_s was greater than 1, then the school was a certainty school, and E_s was the expected number of hits for the school, with the random integer hits value (either the largest integer less than E_s or the smallest integer greater than E_s) determining the school's student sample size.

We also added the constraint to minimize overlap with the Education Longitudinal Study (ELS). This sample was selected for another National Center for Education Statistics (NCES) study occurring in the same year as NAEP 2002, and the intent was to minimize undue burden on schools resulting from selection in both surveys. The technique used to minimize overlap with ELS is based on the logic of conditional probabilities, as with the Trial Urban District Assessment (TUDA). The conditional measures computed on this particular application to the twelfth-grade public school sample are those measures used to select the schools, minimizing overlap with the ELS while still maintaining the unconditional measures π_s, which are assumed as the correct probabilities in weighting (taking as the sample design the "universal" sample design of all possible ELS and NAEP school samples).

¹ Within school minority strata (high/low) and race/ethnicity strata (Black or Hispanic and other).

Printer-friendly Version