- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
NAEP Technical DocumentationWeighting Procedures for the 2018 Assessment
NAEP assessments use complex sample designs to create student samples that generate population and subpopulation estimates with reasonably high precision. School and student sampling weights ensure valid inferences from the school and student samples to their respective populations. In 2018, weights were developed for schools and students sampled at grade 8 for assessments in civics, geography, U.S. history, and technology and engineering literacy (TEL).
Student Weights
The assessments for civics, geography, and U.S. history at grade 8 in both public and private schools were administered in two modes: paper and pencil mode, and a digital mode using tablets.
As described in the
NAEP 2018 sample design section, the random sample of students assigned to the paper-based assessment (PBA) group were assessed using paper and pencil, and the random sample of students assigned to the digitally based assessment (DBA) group were assessed using tablets. Separate sampling weights were computed for the DBA-only sample, PBA-only sample, and DBA and PBA samples combined. The weighting procedures for the civics, geography, and U.S. history assessments described in this report are based on combined DBA/PBA samples. The TEL assessment at grade 8 for public and private schools was only administered to students via tablets and therefore involved the computation of only one set of weights.
Each student was assigned a weight to be used for making inferences about students in the target population. This weight is known as the final full-sample student weight, and it contains six major components:
- the student base weight,
- school nonresponse adjustment,
- student nonresponse adjustment,
- school weight trimming adjustment,
- student weight trimming adjustment, and
- student raking adjustment.
The student base weight is the inverse of the overall probability of selecting a student and assigning that student to a particular assessment. The sample design that determines the base weights is discussed in the NAEP 2018 sample design section.
The student base weight is adjusted for two sources of nonparticipation: school level and student level. These weighting adjustments seek to reduce the potential for bias from such nonparticipation by
- increasing the weights of students from schools similar to those schools not participating, and
- increasing the weights of participating students similar to those students from within participating schools who did not attend the assessment session (or makeup session) as scheduled.
Furthermore, the final student weights reflect the
trimming of extremely large weights at both the school and student levels. These weighting adjustments seek to reduce variances of survey estimates.
An additional weighting adjustment was implemented in the geography and U.S. history assessments so that estimates for key student-level characteristics were in agreement across those assessments. This adjustment was implemented using a
raking procedure.
In addition to the final full-sample student weight, a set of
replicate weights was provided for each student. These replicate weights are used to calculate the variances of survey estimates using the
jackknife repeated replication method.
The methods used to derive these weights were aimed at reflecting the features of the sample design. When the jackknife variance estimation procedure is implemented, approximate unbiased estimates of sampling variance are obtained. In addition, the various weighting procedures were repeated on each set of replicate weights to appropriately reflect the impact of the weighting adjustments on the sampling variance of a survey estimate.
School Weights
In addition to student weights, school weights were calculated to provide secondary users means to analyze data at the school level. The school weights are subject specific and are computed for the schools that contained at least one student that participated in the NAEP assessment for that subject.
Each school was assigned a weight to be used for making inferences about schools in the target population. This weight, known as the final full-sample school weight, contains five major components:
- the school base weight,
- school nonresponse adjustment,
- school weight trimming adjustment,
- school-session assignment weight, and
- small-school subject adjustment.
The school base weight is the inverse of the overall probability of selecting a school for a particular assessment.
The school nonresponse adjustment increases the weights of participating schools to account for similar schools that did not participate, and the school trimming adjustment reduces extremely large weights to decrease variances of survey estimates. These two adjustments are the same school-level adjustments used in the final full-sample student weight described above.
The school-session assignment weight reflects the probability that the particular session type was assigned to the school.
The small-school subject adjustment accounts for very small schools that did not have enough participating students for every subject intended for the school. School weights for subjects that had at least one eligible student are inflated by this factor to compensate for schools of the same size that did not have any eligible students for those subjects and would not be represented otherwise.
In addition to the final full-sample weight, a set of replicate weights was provided for each school. These replicate weights are used to calculate the variances of school-level estimates using the jackknife repeated replication method.
Quality Control Procedures
Quality control checks were implemented throughout the weighting process to ensure the accuracy of the full-sample and replicate school and student weights. See the Quality Control on Weighting Procedures link above for the various checks implemented and main findings of interest.
Last updated 13 December 2022 (PG)
