Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 17 Mathematics Long-Term Trend Bridge Study: 2004
Range of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 17 mathematics long-term trend bridge study: 2004
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa
M1 N251101 1 - 2 900 98 0.95
N256101 1 - 2 1,000 100 0.96
N260601 1 - 2 1,000 99 0.96
N263001 1 - 2 900 99 0.97
N264301 1 - 2 700 97 0.92
N278501 1 - 2 800 98 0.96
N278502 1 - 2 800 98 0.96
N278503 1 - 2 800 98 0.95
N287301 1 - 2 700 97 0.95
N287302 1 - 2 800 96 0.93
M2 N255801 1 - 2 700 99 0.97
N259001 1 - 2 800 97 0.93
N260801 1 - 2 800 99 0.98
N263101 1 - 2 900 99 0.98
N280401 1 - 2 900 99 0.97
M21 N307301 1 - 2 900 99 0.98
N307701 1 - 2 800 100 1.00
N308101 1 - 2 900 99 0.99
N308501 1 - 2 800 99 0.98
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education