NAEP Technical DocumentationNAEP Assessment Weighting Procedures

Since each selected school that participates in the assessment effort and each student assessed constitute only a portion of the full population of interest, weights are applied to both schools and students. The weights permit valid inferences to be drawn from the student samples about the respective populations from which they were drawn and, most importantly, ensure that the results of the assessments are fully representative of the target populations. Note that this documentation of weights describes only weighting components that pertain to analyses conducted at the student level, since that is the approach used in the analysis and reporting of NAEP results.

The final weights assigned to each student as a result of the estimation procedures are the product of the following steps:

• assignment of a school and student base weight, the reciprocal of the overall initial probability of selection;
• adjustment of the school base weights to reduce extreme variability, arising from special circumstance;
• adjustments for school and student nonresponse;
• adjustment (if needed) to reflect assignment to a specified assessment subject; and
• adjustment of the student weights to reduce variability by benchmarking to known student counts obtained from independent sources, such as the Census Bureau (this procedure only applies to some NAEP assessments).

School base weights are assigned separately by grade and, as noted, are the reciprocal of the school’s probability of selection for that grade.

Each sampled student receives a student base weight, whether or not the student participates in the assessment process. The base weight reflects the number of students that the sampled student represents in the population of interest. The sum of the student base weights for a given subgroup provides an estimate of the total number of students in that subgroup.

Since nonresponse is unavoidable in any survey of a human population, a weighting adjustment is introduced to compensate for the loss of sample data and to improve the accuracy of the assessment estimates. Nonresponse adjustments are applied at both the school and the student levels; the weights of responding schools are adjusted to reflect the nonresponding schools, and the weights of responding students, in turn, receive an adjustment to account for nonresponding students.

Nonresponse bias is kept to a minimum by creating nonresponse adjustment classes based on characteristics associated with achievement on NAEP assessments, as reflected in historical NAEP data.

The complexity of the sample selection process as well as the variations in school enrollment can result in extremely large weights for both schools and students. Since unusually large weights are likely to produce large sampling variances for statistics of interest, and especially so when the large weights are associated with sample cases reflective of rare or atypical characteristics, such weights usually undergo an adjustment procedure that trims or reduces extreme weights. Here, the motivation is to improve the precision of the survey estimates.

Estimates of the sampling variance of statistics derived from a NAEP assessment are computed using a replication method known as jackknife. This process of replication involves the repeated selection of portions of the sample (replicates). A separate set of weights is produced for each replicate, using the same weighting procedures as for the full sample. The replicate weights, in turn, are used to produce estimates for each replicate (replicate estimates). The variability among the calculated replicate estimates is then used to obtain the variance of the full-sample estimate. More detail about the jackknife replicated variance estimation procedure used in NAEP is available in Replicate Variance Estimation.

Detailed discussions of the specific sample selection, weighting, and variance estimation procedures are included in the documentation for the specified year of interest.

Printer-friendly Version