Items with responses that do not appear to fit the item response models used to describe the data may be treated in special ways. Rarely, items with responses that do not fit the models may be deleted from NAEP scales. For instance, dichotomous items with responses that are non-monotonic would be deleted from NAEP scales. These are items where the likelihood of a correct response is lower for students who do well on the other items than for students who do poorly on the other items. Other items that might be deleted from NAEP scales are those that can not be modified to fit the item response models and that have empirical and theoretical item response functions that vary a great deal.
Information about the way specific items are treated in NAEP scales is carefully tracked. Related items might be combined into "cluster items" and treated as a single item during scaling. Related items can be identified by test developers when items are written or scored, or by analysis staff when item responses are prepared for scaling or when items are scaled. Combining two or more items into a single item ameliorates dependencies among the items, making the assumption of local independence more realistic.
Polytomous items with categories of response data that do not fit the generalized partial credit model when empirical and theoretical item response functions are compared may fit the model more closely if categories of response data are combined. Therefore, some categories of polytomous items might be recorded to reflect the combination of response categories.
Finally, when items that were administered in more than one assessment year have responses that differ for those years, items might be scaled separately across assessment years. These items are most often identified by empirical item response functions for the different assessment years that differ markedly from one another and from the overall theoretical item response function.
|2001||Geography R3 / R2|
|U.S. history R3 / R2|
|2000||Mathematics R3 / R2|
|Reading R3 / R2|
|Science R3 / R2|
NOTE: R2 refers to accommodations not offered. R3 refers to accommodations offered to students with disabilities (SD) and/or English language learners (ELL) participating in the NAEP assessment.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Various years, 2000–2008 Assessments.
|2008||Mathematics long-term trend|
|Reading long-term trend|
|2004||Mathematics long-term trend|
|Reading long-term trend|
|SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Long-term Trend Assessments 2004 and 2008.|