NAEP Technical DocumentationStudent Sample Selection for the 2007 State Assessment

Within each sampled school, a sample of students was selected from a listing of the students in the grade so that every student had an equal chance of selection. The student lists were submitted in multiple ways. E-filing is an electronic submission system. Excel files are submitted for sampled schools by school coordinators or NAEP State Coordinators. Files can be submitted for one school at a time or for an entire jurisdiction at once. This method allows schools to easily submit student demographic data electronically with the student lists, easing burden on NAEP field supervisors and school coordinators. Schools that are unable to submit their student lists using the e-filing system provide hard-copy lists via the student listing form to NAEP field supervisors. In 2007, there were 14,714 schools that e-filed their student lists, while 999 lists were submitted using the student listing form.

In year-round, multi-track schools, students who were not scheduled to be in school on the assessment day were removed from the student lists prior to sampling. Student base weights were adjusted to account for these students.

The sampling process was the same regardless of list submission type. The sampling process was systematic (e.g., if the sampling rate was one-half, a random starting point of one or two was chosen, and every other student on the list was selected). For e-filed schools only, where demographic data was submitted for every student on the frame, students were sorted by gender and race before the sample was selected to implicitly stratify the sample.

In the certainty jurisdictions, all students were sampled in all schools. Otherwise, the sample sizes were as follows:

62 fourth-graders,
93 eighth-graders in jurisdictions participating in writing,
63 eighth-graders in jurisdictions not participating in writing, and
30 students for grade 4 and 8 schools in Puerto Rico.

Larger schools may have been selected more than once in the sampling process and thus may have a larger sample size. In addition, most fourth-grade schools chose the option of taking all students when enrollment was less than 120. This increased the fourth-grade sample size in many states beyond the designated target.

A pilot test was conducted in the largest 35 schools in Puerto Rico. In these schools, twice the number of students were selected, with one-half being assigned to the mathematics assessment and one-half to the pilot test.

Some students enrolled in the school after the sample was selected. In such cases, new enrollees were sampled at the same rate as the students on the original list.

In fourth grade, students were randomly assigned to a mathematics or reading assessment, with one-half probability of getting each subject. In eighth-grade schools that did not participate in the state writing sample, the sampled students were randomly assigned to reading, mathematics, and writing as follows: 30 for mathematics, 30 for reading, and 3 for writing. In eighth-grade schools that did participate in the state writing assessment, students were randomly assigned to a mathematics, reading, or writing assessment, with a one-third probability of getting each subject. This was implemented by spiraling: the booklets assigned to sampled students were provided from booklet packets which had, on average, equal numbers of each of the relevant assessments in a randomized order.

Some of the students who were English language learners or who had a disability were excluded from the assessment because they could not be assessed with the accommodations NAEP provides. Excluded students were removed from the sample.

Last updated 13 April 2010 (GF)

Printer-friendly Version