Skip to main content
Skip Navigation

Table of Contents | Search Technical Documentation | References

NAEP Analysis and ScalingEstimation of NAEP Score Scales → Treatment of Items in NAEP to Avoid Violations of Assumptions

NAEP Technical DocumentationTreatment of Items in NAEP to Avoid Violations of Assumptions


Treatment of Items in NAEP Assessments

Results of NAEP DIF Analyses

NAEP provides information about the treatment of specific items in NAEP scales and about evidence of differential item functioning (DIF) for specific items. Using current methodologies in psychometrics, the assumption of conditional independence and the assumption that the data fit the Item Response Theory (IRT) models are examined and controlled in NAEP in several ways. They are examined by considering the results of DIF analyses, item fit statistics, and plots of empirical and theoretical item response functions. They are controlled by treating missing and "not reached" responses in reasonable ways, maintaining the context and administration of items across assessments, collapsing categories of polytomous items when appropriate, combining two or more items into a single item (cluster item), or making decisions about the inclusion or deletion of an item in a scale based on data. The identification and amelioration of violations of IRT assumptions is an area of ongoing research in educational measurement. For example, studies have investigated local item dependence (Habing and Donoghue 1998Yen 1993), assessing the fit of the item response function (Donoghue and Hombo 1999Hombo, Donoghue, and Thayer 2000Orlando and Thissen 2000) and item parameter drift (Donoghue and Isham 1998), and detecting and describing multidimensionality (e.g., Roussos, Stout, and Marden 1998Zhang and Stout 1999).

Last updated 02 November 2022 (SK)