​NAEP Technical DocumentationWeighting Procedures for the 2019 Assessment

			Computation of Full-Sample Student Weights Computation of Replicate Student Weights for Variance Estimation Computation of Full-Sample School Weights Computation of Replicate School Weights for Variance Estimation Quality Control on Weighting Procedures

NAEP assessments use complex sample designs to create student samples that generate population and subpopulation estimates with reasonably high precision. School and student sampling weights ensure valid inferences from the student samples to their respective populations. In 2019, weights were developed for school and students sampled at grades 4, 8, and 12 for assessments in mathematics, reading, and science.

Student Weights

The assessments for mathematics and reading at grades 4 and 8 in both public and private schools were administered in a digital mode using tablets for all students. As described in the NAEP 2019 sample design section, for the mathematics and reading assessments at grade 12 and science assessment at grades 4, 8, and 12, students were assigned to either the digitally based assessment (DBA) or the paper-based assessment (PBA) during student sample selection. The random sample of students assigned to the PBA group were assessed using paper and pencil, and the random sample of students assigned to the DBA group were assessed using tablets. Separate sampling weights were computed for the DBA-only sample, PBA-only sample, and DBA and PBA samples combined using the same weighting procedures. The weighting procedures for mathematics and reading described in this report are based on combined DBA/PBA samples.

Each student was assigned a weight to be used for making inferences about students in the target population. This weight is known as the final full-sample student weight and contains the following major components:

• the student base weight,
• school nonresponse adjustments,
• student nonresponse adjustments,
• school weight trimming adjustments,
• student weight trimming adjustments, and
• student raking adjustment.

The student base weight is the inverse of the overall probability of selecting a student and assigning that student to a particular assessment. The sample design that determines the base weights is discussed in the NAEP 2019 sample design section.

The student base weight is adjusted for two sources of nonparticipation: at the school level and at the student level. These weighting adjustments seek to reduce the potential for bias from such nonparticipation. Responding schools receive a weighting adjustment to compensate for nonresponding schools, and responding students receive a weighting adjustment to compensate for nonresponding students.

Furthermore, the final weights reflect the trimming of extremely large weights at both the school and student level. These weighting adjustments seek to reduce variances of survey estimates.

An additional weighting adjustment was implemented in the state and Trial Urban District Assessment (TUDA) samples so that estimates for key student-level characteristics were in agreement across assessments in reading and mathematics. This adjustment was implemented using a raking procedure. A similar but separate adjustment was also implemented for the national public school science samples at grades 4 and 8. The raking procedure implemented for science brought estimates for key student-level characteristics into agreement with those from mathematics and reading at the national level.

In addition to the final full-sample weight, a set of replicate weights was provided for each student. These replicate weights are used to calculate the variances of survey estimates using the jackknife repeated replication method. The methods used to derive these weights were aimed at reflecting the features of the sample design, so that when the jackknife variance estimation procedure is implemented, approximately unbiased estimates of sampling variance are obtained. In addition, the various weighting procedures were repeated on each set of replicate weights to appropriately reflect the impact of the weighting adjustments on the sampling variance of a survey estimate. A finite population correction (fpc) factor was incorporated into the replication scheme so that it could be reflected in the variance estimates for the reading and mathematics assessments. See Computation of Replicate Student Weights for Variance Estimation for details.

In addition to student weights, school weights were calculated to provide secondary users means to analyze data at the school level. The school weights are subject specific and represent the schools that contained at least one student that participated in the NAEP assessment for that subject.

Each school was assigned a weight to be used for making inferences about schools in the target population. This weight is known as the final full-sample school weight, and it contains five major components:

• the school base weight,
  
• school nonresponse adjustment,
• school weight trimming adjustment,
• school session assignment weight, and
• small-school subject adjustment.

The school base weight is the inverse of the probability of selecting a school for a particular assessment. The school nonresponse adjustment increase the weights of participating schools to account for similar schools that did not participate, and the school trimming adjustment reduce extremely large weights to decrease variances of survey estimates. These two adjustments are the same school-level adjustments used in the student full-sample weight described above.

The school session assignment weight reflects the probability that the particular session type was assigned to the school.

The small-school subject adjustment accounts for very small schools that did not have enough participating students for every subject associated to the school. School weights for subjects that had at least one eligible student are inflated by this factor to compensate for subject(s) did not have any eligible students in that school and, thus, not represented. In addition to the full-sample weight, a set of replicate weights was provided for each school. The school replicate weights are used to calculate the variances of school-level survey estimates using the jackknife repeated replication method.

Quality control checks were carried out throughout the weighting process to ensure the accuracy of the full-sample and replicate weights. See Quality Control on Weighting Procedures for the various checks implemented and main findings of interest.

Printer-friendly Version