Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 17 Mathematics Long-Term Trend Assessment: 2004
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 17 mathematics long-term trend assessment: 2004
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa
M21 N307301 1–2 900 99 0.97
N307701 1–2 900 100 0.99
N308101 1–2 900 99 0.98
N308501 1–2 900 98 0.96
M22 N302901 1–2 900 99 0.93
N308901 1–2 900 99 0.98
N309301 1–2 900 99 0.97
N309901 1–2 900 99 0.98
N310201 1–2 900 100 0.99
M23 N316701 1–2 900 100 0.98
N318901 1–2 900 99 0.97
N319301 1–2 900 100 1.00
N319302 1–2 900 99 0.99
N319303 1–2 900 100 1.00
N320101 1–2 900 100 1.00
N320301 1–2 800 98 0.97
N320302 1–2 800 99 0.97
N320401 1–2 800 100 0.99
N320601 1–2 800 99 0.97
M24 N315501 1–2 800 99 0.98
N321001 1–2 900 99 0.99
N321101 1–2 800 100 1.00
N321401 1–2 700 100 0.99
N321901 1–2 700 100 0.99
MX51 N303801 1–2 900 100 0.99
N304101 1–2 900 100 1.00
N304201 1–2 900 99 0.96
N304601 1–2 900 99 0.99
N304901 1–2 900 100 1.00
MX52 N299001 1–2 900 99 0.98
N305501 1–2 900 99 0.99
N305801 1–2 900 100 0.99
N306201 1–2 800 98 0.96
N306401 1–2 900 100 0.99
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education