- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Technical DocumentationState NAEP School and Student Nonresponse Adjustments
Nonresponse is very common in surveys of human populations. Nonresponse leads to the loss of sample data that must be compensated for in the weights of the responding sample members. The purpose of the nonresponse adjustments is to reduce the mean square error of survey estimates. While the nonresponse adjustment reduces the bias from the loss of sample, it also increases variability among the survey weights leading to increased variances. However, it is presumed that the reduction in bias more than compensates for the increase in the variance, thereby reducing the mean square error of survey estimates. Nonresponse bias is adjusted for in the NAEP surveys at both the school and the student levels: the responding schools receive a weighting adjustment to compensate for nonresponding schools, and responding students receive a weighting adjustment to compensate for nonresponding students.
The paradigm used for nonresponse adjustment in NAEP is the quasi-randomization approach (Oh and Scheuren 1983). In this approach, school response cells are based on characteristics of schools known to be related to both response and to achievement levels, such as the socioeconomic status of the school. Likewise, student response cells are based on characteristics of the schools containing the students, and student characteristics, which are known to be related to both response and to achievement levels, such as student race, gender, and age.
Under this approach, sample members are assigned to mutually exclusive and exhaustive response cells based on pre-determined characteristics. The weighting adjustment for each responding unit will be equal to the summation of adjusted base weights for all units divided by the summation of adjusted base weights for all responding units. In this way, the weights of responding units in the cell are "weighted up" to represent the full set of responding and nonresponding units in the response cell. The weights used in this process are generally the product of the school base weight and the estimated grade enrollment for the school for school nonresponse adjustment, and the student base weight for student nonresponse adjustment. This student base weight generally includes a school nonresponse adjustment.
The quasi-randomization paradigm views nonresponse as another stage of sampling. Within each nonresponse cell, the paradigm assumes that the responding sample units are a simple random sample from the total set of all sample units. If this model is valid, then the use of the quasi-randomization weighting adjustment will eliminate any nonresponse bias. Even if this model is not valid, the weighting adjustments will eliminate bias if the achievement scores are homogeneous within the response cells (i.e., bias is eliminated if there is homogeneity either in response propensity or in achievement levels). See, for example, chapter 4 of Little and Rubin (1987).
Last updated 09 December 2008 (EH)
