Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Estimation of NAEP Score Scales → Checks for Violations of Assumptions → Comparisons of Empirical and Theoretical Item Response Functions → Items Showing Good Item Fit → Dichotomous Item With Data for Two Assessment Years

NAEP Technical DocumentationDichotomous Item With Data for Two Assessment Years

When the same items are presented in two assessment years, the empirical item response functions (IRFs) for the two years can be compared to one another and to the theoretical IRF. Normally, the curves for the two years differ somewhat due to the sampling of students for each year. This figure contains a plot for a dichotomous item from the 1994 and 1998 grade 8 national main reading assessments.

In the plot, the horizontal axis represents the provisional theta () scale with mean 0 and standard deviation 1, while the vertical axis represents the probability of a correct response. The solid curve is the theoretical IRF based on the item parameter estimates and the equation for the two-parameter IRT model. The centers of the symbols represent the empirical proportions correct. The sizes of the symbols are proportional to the sum of the posteriors at each point on the scale for all of those who received the item; this is related to the number of students contributing to the estimation of that empirical proportion correct.

Dichotomous item exhibiting good model fit across assessment years

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Red inverted triangles represent 1998 grade 8 reading assessment data; green triangles represent 1994 grade 8 reading assessment data. The curve indicates the estimated IRF assuming a logistic form. This item is a constructed-response item scored as "correct" or "incorrect."

Printer-friendly Version