NAEP Technical DocumentationSampling of Public Schools for the 2008 Long-Term Trend (LTT) Assessment

When designing each school sample, there are five objectives that underlie the process of determining the probability of selection for each school and how many students are to be sampled from each selected school:

• to meet the target student sample size for each age-specific sample;
• to select an equal-probability sample of students;
• to limit the number of students that are selected from a school;
• to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included; and
• to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools.

The goal in determining the school's measure of size (MOS) is to optimize across the last four objectives in terms of maintaining the accuracy of estimates and the cost effectiveness of the sample design.

Therefore, to meet the target student sample size objective and achieve a reasonable compromise among the other four objectives, the following algorithm was used to assign a measure of size to each school based on its age-specific enrollment as estimated from the sampling frame data.

The measures of size vary by enrollment size and within school target sample size. In general, very small schools were assigned an MOS that is one quarter of the target sample size of 62 students. The preliminary measures of size MOS_js) for age j in school s were set as follows:

The next task in this development is to describe b_j, the constant of proportionality for each grade. It is a sampling parameter that, when multiplied with a school’s preliminary MOS_js, yields the school’s final measure of size. It is computed in such a way that, when used with the systematic sampling procedure, the target student sample size is achieved. For public schools, b_j is 0.000073 for age 9, 0.000079 for age 13, and 0.000106 for age 17.

The final measure of size, E_js, is defined as: 

The quantity u_j (the maximum number of times a school can be selected or “hits” allowed) in this formula is designed to put an upper bound on the burden for the sampled schools. For public schools, u_j is 1.

In addition, new- and newly-eligible-schools were sampled from the new-school frame. The assigned measures of size for these schools

used the b_j and u_j values from the main school sample. The variable is the probability of selection of the district d into the new-school district sample.

Schools were ordered within each jurisdiction using the serpentine sort described under the stratification of public schools. A systematic sample was then drawn using this serpentine sorted list and the measures of size. The numbers of public schools selected, including the original and new school sample were approximately 320, 360, and 400 for ages 9, 13, and 17, respectively.

The estimated number of age-eligible students was computed using a derived linear function of estimated grade enrollments, with the coefficients for this function derived from current population survey (CPS) data. For 9-year-olds, the frame contained schools with a second, third, fourth, or fifth grade. M_js(2), M_js(3), M_js(4), and M_js(5) denote the estimated enrollments for the four grades respectively within each school s for age sample j, and F_j(2), F_j(3), F_j(4), and F_j(5) denote the global proportion of students who were 9-years-old (birth months January 1998 through December 1998) within each grade, Census region, and within public schools. These values are shown in the table below for public schools. Age Distribution Fractions for Ages 9 and 13 describes how proportions are computed.

The measure of size for the 9-year-olds component of the long-term trend (LTT) study for each school was computed as follows:

For 13-year-olds, the frame contained schools with a fifth, sixth, seventh, or eighth grade. M_js(6), M_js(7), M_js(8), and M_js(9) denote the estimated enrollments for the four grades respectively within each school s for age sample j, and G_j(6), G_j(7), G_j(8), and G_j(9) denote the global proportion of students who were 13-years-old (birth months January 1994 through December 1994) within each grade, Census region, and within public schools. These values are shown in the table below for public schools. Age Distribution Fractions for Ages 9 and 13 describes how sample design staff computed these proportions.

The measure of size for the 13-year-olds component of the LTT study for each school was computed as follows:

For the age 17 component, the frame contained schools with a ninth, tenth, eleventh, or twelfth grade. M_js(9), M_js(10), M_js(11), and M_js(12) denote the estimated enrollments for the four grades respectively within each school s for age sample j, and J_j(9), J_j(10), J_j(11), and J_j(12) denote the global proportion of students who were 16 years old (birth months October 1990 through September 1991) within each grade and Census region. Current Population Survey (CPS) School Supplement (October 2005) was utilized to provide the proportions within each of ninth, tenth, eleventh, and twelfth grades for the 16-years-olds (as of October 1, 2005). The tables are identical for private and public schools since the CPS does not provide information by type of school. It was decided to use the CPS in lieu of NAEP 2005 due to the small number of private schools in the twelfth grade sample.These values are shown in the table below for public schools.The derivation of the Age Distribution Fractions for Age 17 was much more straightforward.

The measure of size for the age 17 component of the LTT study for each school was computed as follows:

Printer-friendly Version