Table of Contents  |  Search Technical Documentation  |  References

NAEP Sample Design → NAEP 2003 Sample Design → 2003 State Assessment Sample Design → Substitute Schools for the 2003 State Assessment

NAEP Technical DocumentationSubstitute Schools for the 2003 State Assessment

			Selecting Substitutes for the State Assessment

In anticipation of school-level nonresponse, substitute schools were selected for many of the original sample schools within each jurisdiction. The theory behind using substitutes is that replacing a nonresponding original sample school with another school can be anticipated as yielding similar assessment results. Using the substitute, thus, would reduce nonresponse bias. The decision as to whether to activate any or all of the substitutes for nonresponding schools in a given jurisdiction was made by the NAEP state coordinator, taking account of the response rate and the burden on schools. (Only one substitute school was activated for public schools in 2003.) Otherwise, nonresponse adjustment alone was used to correct for nonparticipation.

Substitute schools were selected by comparing each sampled school to each school not in the original school sample for the same grade, the same state, and the same urbanicity stratum. (Also excluded from the list of potential substitutes were all new schools, because of inadequate information to pair to an original, and all closed/ineligible schools identified as such in the new-school canvassing process.) A distance measure was generated for each potential school pair, based on the percentage of students in the most prevalent minority and the next most prevalent minority within the urbanicity stratum (as used for the stratification of schools). The distance measure also incorporated the estimated grade enrollment and the achievement score (in jurisdictions for which it was available) or median income (in jurisdictions for which achievement score was not available). Only pairs of schools with small distance measures were selected to be original-substitute pairs. This procedure assured that substitutes would be in the same state and urbanicity stratum, and would have minority percentages, grade enrollment, and achievement scores (or median income) close to that of the original school.

In many cases, no substitute candidates were available for a sampled school. The first choice for a substitute was a school out-of-district (i.e., in a different district from the original school). If no such substitute was available, a substitute was allowed within the district.

Printer-friendly Version