NAEP Technical DocumentationUsing Plausible Values to Estimate Group Scale Score Statistics and Their Variances

One way to estimate

is with random draws from the conditional distributions p (θ_i|x_i,y_i), which are obtained for all respondents by the method described in Using Population-Structure Model Parameters to Create Plausible Values for Easy Computation. Let θ_m be the m^th such vector of plausible values, consisting of a multidimensional value for the latent variable of each respondent. This vector is a plausible representation of what the true θ vector might have been, had we been able to observe it.

The following steps describe how an estimate of a scalar statistic t(θ, y) and its sampling variance can be obtained from M (>1) such sets of plausible values. (Five sets of plausible values are used in NAEP analyses.)

Using each set of plausible values _m in turn, evaluate t as if the plausible values were true values of θ. Denote the results _m, for m = 1, ... , M.
Using the jackknife variance estimator, compute the estimated sampling variance of _m, denoting the result U_m .
The final estimate of t is
Compute the average sampling variance over the M sets of plausible values, to approximate uncertainty due to sampling respondents
Compute the variance among the M estimates _m, to approximate the between-estimate variance
The final estimate of the variance of t* is the sum of two components

In this equation, (1 + M ^-1)B is the estimate of variance due to the latency of θ. Due to the excessive computation that would be required, NAEP analyses do not compute and average jackknife variances over all five sets of plausible values, but use that computed from the first set. Thus, in NAEP reports, U* is approximated by U₁.

Last updated 06 February 2008 (GF)

Printer-friendly Version