Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Estimation of Population and Student Group Distributions → The Role of IRT Models and Population-Structure Models in NAEP

The Role of IRT Models and Population-Structure Models in NAEP

The goal of NAEP is to estimate the distributions of scale scores for groups of students. Because there is no way to know the underlying performance for any student exactly, NAEP uses every item response for students in every group to estimate the distribution of scale scores for groups. To build the models necessary to do this, consider the probability that a particular student has an underlying performance level of theta. Given everything we know about the student, this probability is of the form:

The probability of vector theta given vector x, vector y, matrix alpha, matrix gamma, and matrix sigma

where theta is the vector consisting of the k underlying performance levels for the k subscales in the subject area, x is the vector consisting of the vectors of item responses, y is the vector specifying the group membership, α represents the parameters in the IRT models, and matrix gamma and matrix sigma are parameters of the population structure model.

This distribution can be separated into two parts: the distribution of the item responses conditional on the underlying performance level of the student, and the distribution of underlying performance conditional on which students are in the group.

The probability of vector theta given vector x, vector y, matrix alpha, matrix gamma, and matrix sigma is proportional to the probability of vector x given vector theta, vector y, matrix alpha, matrix gamma, and matrix sigma times the probability of vector theta given vector y, matrix alpha, matrix gamma, and matrix sigma, which equals the probability of vector x given vector theta and matrix alpha times the probability of vector theta given vector y, matrix gamma, and matrix sigma

The first of the two parts is modeled using the Item Response Theory (IRT) measurement models; the second is modeled using a population-structure model. The first of the two parts, the distribution of the item responses conditional on the underlying performance level of the student, does not depend on group membership at all, because IRT models require the assumption that underlying performance level only depends on the item responses for a student and the item parameters. The IRT measurement models are used to define the NAEP score scales.

Go to a description of the Estimation of NAEP Score Scales

Go to a description of NAEP Population-Structure Models

Last updated 08 August 2008 (RF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education