Table of Contents  |  Search Technical Documentation  |  References

NAEP Sample Design → NAEP 2003 Sample Design → 2003 State Assessment Sample Design

NAEP Technical Documentation2003 State Assessment Sample Design

			Target Population Sampling Frame New-School Sampling Frame Stratification of Schools School Sample Selection Trial Urban District Assessment Fourth-Grade Charter School Sample Substitute Schools Ineligible Schools Student Sample Selection School Participation Student Participation

The NAEP 2003 state assessment in mathematics and reading sampled jurisdictions comprising states, U.S. territories, Bureau of Indian Affairs schools, Department of Defense schools, and school districts chosen for the Trial Urban District Assessment. Each sample was designed to produce aggregate estimates with approximately equal precision for all the participating jurisdictions, as well as estimates for various student populations of interest. The target sample in each jurisdiction participating in the assessment was 6,510 students per grade. With a general target of 62 sampled students per school, approximately 105 schools were needed per participating jurisdiction in each grade.

The Bureau of Indian Affairs schools declined participation in the state-level assessment, as well as American Samoa (for grade 4), Guam, and the Northern Mariana Islands. Puerto Rico participated only in the mathematics state assessment (not in the reading state assessment).

The target population for the 2003 state assessment included students in public schools who were enrolled in the fourth and eighth grades at the time of assessment. The sampling frame included public schools having the relevant grade in each jurisdiction. The samples were selected based on a two-stage sample design: selection of schools within participating jurisdictions, and selection of students within schools. The first-stage samples of schools were selected with probability proportional to a measure of size based on the estimated grade-specific enrollment in the schools.

As part of the selection process, schools were hierarchically stratified by charter school status, urbanization, and minority class. Charter school status was the first stratification variable used in 2003 because of a requirement for a charter school oversample for 2003. This is contrasted with 2002, in which the first stratification variable in the hierarchy was whether the school was located in a large or small district. Within minority strata, schools were sorted by state achievement data, where such data were available, and by median household income of the ZIP Code area where the school was located, where state achievement data were not available. Achievement data were supplied by the jurisdictions themselves. Median income data were obtained from the 1990 Census. Other stratification variables were obtained from the National Center for Education Statistics' Common Core of Data (CCD). 

From the stratified frame of public schools for each grade within each jurisdiction, a systematic random school sample of about 100 grade-eligible schools was drawn with probability proportional to a measure of size based on the estimated grade-specific enrollment of the school. One or more sessions were sampled within each school, although, in contrast to the 2002 NAEP design, there was only one session type. The number of sessions selected depended on the school's estimated grade-specific enrollment, though the great majority of schools at grade 4 were allocated a single session.

Ten large urban districts were selected for participation in the Trial Urban District Assessment (TUDA) study, but only nine participated. Albuquerque, New Mexico declined to be oversampled as a TUDA, but did participate in the state-level assessment. Additional schools were sampled from these ten districts at the same time schools were selected for the jurisdiction samples. In addition, a national sample of charter schools was selected for grade 4 in 2003. These subsamples (TUDA and charter schools) affected the design of the state samples in those states where TUDA districts and charter schools were oversampled. In some states, there were distinct sampling rates for each TUDA district, the fourth grade charter schools, and the rest of the state. For any school that belongs to more than one of the groups for which different sampling rates were set, the school was sampled at the highest applicable rate.

Each selected school provided a list of eligible enrolled students, from which a systematic sample of students was drawn. Sixty-two students, if possible, were selected from each school (31 students for mathematics and 31 students for reading). This departure from past NAEP history (i.e., a sample of 60 students per school) was brought about by the spiraling of pilot test booklets with operational assessment booklets in the same session in 2003 assessment.

Printer-friendly Version