Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Estimation of NAEP Scale Scores → Checks for Violations of Assumptions

Checks for Violations of Assumptions

     Horizontal Line     

Differential Item Functioning (DIF)

Dimensionality Studies

Item Fit Statistics

Comparisons of Empirical and Theoretical Item Response Functions

A number of checks are made to detect serious violations of the assumptions underlying the models employed by NAEP. Checks are made to detect multidimensionality of the construct being measured and certain conditional dependencies. DIF analyses are used to examine issues of dimensionality, and what are called chi square statistics in the IRT literature are used to flag responses with serious departures from the IRT model. The latter statistics might better be called item fit statistics, since they do not really have chi square distributions. These checks include comparisons of empirical and theoretical item response functions to identify items for which the IRT model may provide a poor fit to the data. When warranted, remedial efforts, such as collapsing categories of polytomous items or combining two or more items into a single item, are made to mitigate the effects of such violations on inferences. These procedures are used for all items regardless of block format (e.g., passage-based items, discrete items) or response type (e.g., multiple-choice items, constructed-response items).

Last updated 19 November 2009 (GF)

Printer-friendly Version