​​​NAEP Technical DocumentationNonresponse Bias Analyses for the 2018 Assessment

NCES Statistical Standards call for a nonresponse bias analysis to be conducted for a sample with a response rate below 85 percent at any stage of sampling. Weighted school response rates for the 2018 assessment indicated that there was a need for school nonresponse bias analyses for public school and private school samples for grade 8. At the student level no nonresponse bias analyses were necessary since the student-level participation rates for all reporting groups were above the 85 percent participation threshold.

Analyses were conducted for the Social Sciences assessments and Technology and Engineering Literacy (TEL) assessment. For the Social Sciences assessments, the analysis was conducted for both public and private schools with Civics, Geography, and U.S. History combined into one analysis because each sampled school was assessed in all subjects. The analysis for the TEL assessment was conducted for private schools only.

The procedures and results from these analyses are summarized briefly below. The analyses conducted consider only certain characteristics of schools and students. They do not directly consider the effects of the nonresponse on student achievement, the primary focus of NAEP. Thus, these analyses are not conclusive of either the existence or absence of nonresponse bias on student achievement.

Each school-level analysis was conducted in three parts. The first part of the analysis looked for potential nonresponse bias that was introduced through school nonresponse. The second part of the analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of substitution. The third part of the analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of both school substitution and school-level nonresponse weight adjustments. The characteristics examined were census region, private school type, urbanization classification (four categories based on urban-centric locale), size of school (categorical), size of school (continuous), and race/ethnicity enrollment percentages.

Based on the school characteristics available for the private school samples, there does not appear to be evidence of substantial potential bias resulting from school nonresponse after substitution and non-response adjustment. Please see the NAEP 2018 Non Response Bias Analysis Report for more details.

Printer-friendly Version