Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 13 Reading Long-Term Trend Bridge Study: 2004
Range of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 13 reading long-term trend bridge study: 2004
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa Intraclass correlation
H N001527 1 - 2 200 100 1.00
J N001904 1 - 2 200 89 0.78
L N002804 1 - 2 200 91 0.79
M N003104 1 - 2 200 88 0.67
N N003704 1 - 2 100 94 0.87
O N004303 1 - 2 100 95 0.89
P N004605 1 - 2 200 95 0.87
R21 N021203 1 - 5 200 79 0.88
R22 N023802 1 - 4 200 74 0.73
R23 N021803 1 - 4 200 87 0.91
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Reading Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education