Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

School Trimming

Students were assigned relatively large weights in many cases because of

  • underestimation of the number of eligible students in some schools, which leads to inappropriately low probabilities of selection for those schools, and

  • the presence of large schools—especially high schools—in PSUs with small selection probabilities.

In the latter cases, the maximum permissible within-school sampling rate (determined by the maximum sample size allowed per school; see Student Sample Selection) could be smaller than the desired overall within-PSU sampling rate for students. Large weights arose also because very small schools were sampled with low probabilities, coupled with high levels of nonresponse and the compounding of nonresponse adjustments at various levels.

Students with notably large weights have an unusually large effect on estimates such as weighted means. The variability in weights contributes to the variance of an overall estimate by an approximate factor
1 + CV 2 where CV 2 is the relative variance of the weights. An occasional unusually large weight will likely produce large sampling variances of the statistics of interest, especially when the large weights are associated with students with atypical performance characteristics.

To reduce the effect of large contributions to variance from a small set of sample schools, the weights of such schools were reduced, i.e., trimmed. The trimming procedure introduces a bias but should reduce the mean square error of sample estimates.

The trimming algorithm for school weights was identical to that used since 1996 and trimmed the weight of any school that contributed more than a specified proportion, to the estimated variance of the estimated number of students eligible for assessment.

Let

M = Number of schools in which a specified assessment was conducted
Wi = Weight assigned to school "i" (i.e., πi–1 × SUBADJi × FTADJi)
x prime sub i = Estimated number of grade-eligible students in school "i" (i.e., the sum of the within-school weights, adjusted for nonresponse, for the students assessed)
x double prime sub i = W sub i times x prime sub i
x bar double prime equals 1 over M times summation over i equals 1 to M of x double prime sub i

A rough approximation to the unit variance of the x double prime sub i is

one over M times summation over i of open paren x double prime sub i minus x bar double prime closed paren squared

A trimming method was used that reduced the weight Wi for a small number of schools so that no school makes a contribution to the sum shown above that is greater than a specified proportion θ. That is, for any school "j", the weight Wj, after all weights have been trimmed as required, satisfies the condition

open paren x double prime sub j minus x bar double prime closed paren squared is less than or equal to theta times summation of open paren x double prime sub i minus x bar double prime closed paren squared

The weight is not to be altered if x double prime sub j is less than x bar double prime, and the condition is equivalent to

x double prime sub j minus x bar double prime is less than or equal to the square root of open bracket theta times summation open paren x double prime sub i minus x bar double prime closed paren squared closed bracket

or

W sub j is less than or equal to open paren 1 divided by x prime sub j closed paren times open bracket x bar double prime plus the square root of open bracket theta times summation open paren x double prime  sub i minus x bar double prime closed paren squared closed bracket closed bracket

The school-level trimming was done iteratively. The weight for each school that failed to satisfy the inequality was reduced to the value given by the right-hand side of the inequality by using the initial weights. The procedure was iterated using the weights as trimmed.

The value of θ to be used was chosen by judgment to provide negligible bias while substantially reducing variance. The chosen value of θ was 10/M.

The trimming procedure was done separately within each NAEP region. A total of five schools were trimmed for grade 4, with trimming factors TRIMi of 0.75 to 0.937; a total of four schools for grade 8, with trimming factors TRIMi of 0.882 to 0.959; and a total of two schools for grade 12, with trimming factors TRIMi of 0.776 to 0.917.

The trimmed school weights were computed as πi–1 × SUBADJi × FTADJi × TRIMi.


Last updated 26 August 2008 (FW)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education