​​​​​​​​​​​​​​​​NAEP Technical DocumentationTreatment of Items in NAEP Assessments

Items with responses that do not appear to fit the item response models used to describe the data may be treated in special ways. Rarely, items with responses that do not fit the models may be deleted from NAEP scales. For instance, dichotomous items with responses that are non-monotonic would be deleted from NAEP scales. These are items where the likelihood of a correct response is lower for students who do well on the other items than for students who do poorly on the other items. Other items that might be deleted from NAEP scales are those that cannot be modified to fit the item response models and that have empirical and theoretical item response functions that vary a great deal.

Information about the way specific items are treated in NAEP scales is carefully tracked. Related items might be combined into cluster items and treated as a single item during scaling. Related items can be identified by test developers when items are written or scored, or by analysis staff when item responses are prepared for scaling or when items are scaled. Combining two or more items into a single item ameliorates dependencies among the items, making the assumption of local independence more realistic.

Polytomous items with categories of response data that do not fit the generalized partial credit model when empirical and theoretical item response functions are compared may fit the model more closely if categories of response data are combined. Therefore, some categories of polytomous items might be recoded to reflect the combination of response categories. In some instances, the​ ​​three-parameter logistic model will be applied to items after the score category recoding to achieve a reasonable fit​​.

Finally, when items that were administered in more than one assessment year have responses that differ for those years, items might be scaled separately across assessment years. These items are most often identified by empirical item response functions for the different assessment years that differ markedly from one another and from the overall theoretical item response function.

Treatment of items in NAEP long-term trend assessments: Various years, 2004–2023

Year	Subject area
2022/2023	Mathematics long-term trend
	​ Reading long-term trend
2020	Mathematics long-term trend
	​ Reading long-term trend
2012	Mathematics long-term trend
	Reading long-term trend
2008	Mathematics long-term trend
	Reading long-term trend
2004	Mathematics long-term trend
	Reading long-term trend
NOTE: The block naming conventions used in the 2020 mathematics and reading long-term trend​ assessments are described in the document 2020 Block Naming Conventions in Data Products and TDW . The block naming conventions used in the 2022/2023 mathematics and reading long-term trend​ assessments are described in the document 2022/2023 Block Naming Conventions in Data Products and TDW . In 2020 and 2022/2023, age 17 was not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 2004–2023 Mathematics and Reading Long-Term Trend Assessments.

Printer-friendly Version