Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Estimation of NAEP Score Scales → Checks for Violations of Assumptions → Comparisons of Empirical and Theoretical Item Response Functions → Items Showing Poor Item Fit → Combining the Categories of Constructed-Response Items

NAEP Technical DocumentationCombining the Categories of Constructed-Response Items

When polytomous items show poor fit to the generalized partial credit model, one option is to combine categories of responses. In the first plot, there is a lack of fit for both the unsatisfactory and partial categories for low scale score () values. This is clear from the triangles that do not fall on the solid curves at the left of the plot. There is also a marked misfit for partial and complete response categories in high scale score () values. Triangles not falling on the solid curves at the right of the plot indicate this. After discussion with a subject area expert, the partial and complete response categories of this item were collapsed resulting in the second item plot. The item became a dichotomous item with a fit that is substantially improved.

In both plots, the horizontal axis represents the theta () scale and the vertical axis represents the probability of having a response fall in each category. In the first plot, the solid curves are the theoretical item response functions (IRFs) based on the item parameter estimates and the equation for the generalized partial credit model. The centers of the triangles represent the empirical proportions of students with responses in each category for the 1998 grade 12 reading assessment data.

In the second plot, the centers of the triangles represent the same data, but after the top two categories were combined. The triangles indicate estimated conditional probabilities obtained without assuming a specific model form; the curve indicates the estimated IRF assuming a three-parameter logistic form. Triangles falling directly on the curve indicate good fit.

Printer-friendly Version