Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Reading Combined National and State Main Assessment, by Item and Block: 2007

Range of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 reading combined national and state main assessment, by item and block: 2007
Block Item Range of response codes Sample size Percent agreement Cohen’s Kappa Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
1 New items introduced in 2006 Precalibration that were rescored in 2007.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Reading Assessment.
R3 R017101 1–2 1,500 89.83 0.78
R017102 1–3 1,300 85.68 0.83
R017104 1–3 1,400 88.92 0.88
R017105 1–4 1,400 69.50 0.76
R017107 1–3 1,400 84.86 0.84
R017108 1–2 1,300 96.42 0.87
R017110 1–3 1,300 91.73 0.92
R41 R055501 1–3 1,400 78.95 0.73
R055801 1–3 1,400 82.60 0.67
R056101 1–4 1,400 71.73 0.81
R056301 1–3 1,400 75.34 0.66
R5 R012601 1–2 1,800 87.72 0.75
R012604 1–2 1,500 90.06 0.80
R012607 1–4 1,400 73.74 0.77
R012612 1–2 1,400 87.48 0.74
R6 R013201 1–4 1,500 75.17 0.83
R013203 1–2 1,500 95.58 0.63
R013205 1–2 1,400 96.20 0.80
R013207 1–2 1,400 87.25 0.71
R013209 1–2 1,600 93.97 0.83
R013211 1–2 1,400 86.69 0.73
R013212 1–4 1,300 77.58 0.74
R71 R034601 1–3 1,400 83.54 0.81
R034901 1–3 1,400 84.47 0.85
R035001 1–3 1,400 85.14 0.84
R035201 1–3 1,400 88.30 0.83
R035301 1–4 1,400 70.40 0.84
R8 R053401 1–3 1,500 84.88 0.86
R053402 1–2 1,500 90.55 0.54
R053405 1–4 1,400 70.55 0.73
R053408 1–3 1,500 84.47 0.83
R053410 1–2 1,700 84.04 0.64
R91 R056501 1–2 1,400 94.42 0.87
R056801 1–3 1,400 79.51 0.77
R057101 1–3 1,400 71.26 0.69
R057301 1–3 1,400 69.24 0.72
R10 R013402 1–2 1,700 98.06 0.96
R013403 1–4 1,400 94.14 0.96
R013405 1–2 1,400 92.61 0.85
R013406 1–4 1,400 75.82 0.87
R013407 1–2 1,400 91.90 0.84
R013409 1–2 1,800 91.29 0.82
R013411 1–2 1,400 85.89 0.69
R013413 1–2 1,400 81.73 0.63
R11 R013001 1–2 1,400 88.07 0.71
R013003 1–2 1,400 99.43 0.99
R013004 1–4 1,400 67.58 0.82
R013005 1–2 1,400 86.32 0.67
R013007 1–2 1,300 95.39 0.87
R013008 1–2 1,400 84.88 0.70
R013009 1–2 1,400 87.55 0.67
R013010 1–2 1,400 92.88 0.80
R013011 1–2 1,400 85.76 0.70
R12 R053202 1–3 1,700 83.75 0.79
R053203 1–3 1,600 76.60 0.72
R053205 1–4 1,500 73.26 0.86
R053207 1–3 1,500 78.35 0.72
R053209 1–3 1,500 85.54 0.81
R13 R016201 1–3 1,300 78.50 0.60
R016202 1–3 1,500 83.62 0.76
R016204 1–4 1,200 79.85 0.77
R016205 1–3 1,200 80.77 0.77
R016207 1–3 1,200 92.60 0.94
R016210 1–4 1,200 64.21 0.73
R016211 1–3 1,200 83.05 0.81
R016212 1–3 1,200 87.48 0.89
R016213 1–3 1,200 78.90 0.76
R14 R026501 1–3 1,500 80.66 0.80
R026801 1–4 1,400 75.37 0.83
R027001 1–2 1,300 89.82 0.67
R027201 1–3 1,400 86.34 0.84
R15 R028401 1–3 2,100 85.84 0.84
R028501 1–3 2,000 86.74 0.86
R028801 1–4 1,600 75.78 0.86
R029601 1–3 1,400 81.55 0.79

Last updated 23 June 2010 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education