Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Reading Combined National and State Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 reading combined national and state main assessment, by item and block: 2007
Block Item Range of response codes Sample size Percent agreement Cohen’s Kappa Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
1 New items introduced in 2006 Precalibration that were rescored in 2007.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Reading Assessment.
R3 R017101 1–2 1,500 89.83 0.78 —
R017102 1–3 1,300 85.68 — 0.83
R017104 1–3 1,400 88.92 — 0.88
R017105 1–4 1,400 69.50 — 0.76
R017107 1–3 1,400 84.86 — 0.84
R017108 1–2 1,300 96.42 0.87 —
R017110 1–3 1,300 91.73 — 0.92
R41 R055501 1–3 1,400 78.95 — 0.73
R055801 1–3 1,400 82.60 — 0.67
R056101 1–4 1,400 71.73 — 0.81
R056301 1–3 1,400 75.34 — 0.66
R5 R012601 1–2 1,800 87.72 0.75 —
R012604 1–2 1,500 90.06 0.80 —
R012607 1–4 1,400 73.74 — 0.77
R012612 1–2 1,400 87.48 0.74 —
R6 R013201 1–4 1,500 75.17 — 0.83
R013203 1–2 1,500 95.58 0.63 —
R013205 1–2 1,400 96.20 0.80 —
R013207 1–2 1,400 87.25 0.71 —
R013209 1–2 1,600 93.97 0.83 —
R013211 1–2 1,400 86.69 0.73 —
R013212 1–4 1,300 77.58 — 0.74
R71 R034601 1–3 1,400 83.54 — 0.81
R034901 1–3 1,400 84.47 — 0.85
R035001 1–3 1,400 85.14 — 0.84
R035201 1–3 1,400 88.30 — 0.83
R035301 1–4 1,400 70.40 — 0.84
R8 R053401 1–3 1,500 84.88 — 0.86
R053402 1–2 1,500 90.55 0.54 —
R053405 1–4 1,400 70.55 — 0.73
R053408 1–3 1,500 84.47 — 0.83
R053410 1–2 1,700 84.04 0.64 —
R91 R056501 1–2 1,400 94.42 0.87 —
R056801 1–3 1,400 79.51 — 0.77
R057101 1–3 1,400 71.26 — 0.69
R057301 1–3 1,400 69.24 — 0.72
R10 R013402 1–2 1,700 98.06 0.96 —
R013403 1–4 1,400 94.14 — 0.96
R013405 1–2 1,400 92.61 0.85 —
R013406 1–4 1,400 75.82 — 0.87
R013407 1–2 1,400 91.90 0.84 —
R013409 1–2 1,800 91.29 0.82 —
R013411 1–2 1,400 85.89 0.69 —
R013413 1–2 1,400 81.73 0.63 —
R11 R013001 1–2 1,400 88.07 0.71 —
R013003 1–2 1,400 99.43 0.99 —
R013004 1–4 1,400 67.58 — 0.82
R013005 1–2 1,400 86.32 0.67 —
R013007 1–2 1,300 95.39 0.87 —
R013008 1–2 1,400 84.88 0.70 —
R013009 1–2 1,400 87.55 0.67 —
R013010 1–2 1,400 92.88 0.80 —
R013011 1–2 1,400 85.76 0.70 —
R12 R053202 1–3 1,700 83.75 — 0.79
R053203 1–3 1,600 76.60 — 0.72
R053205 1–4 1,500 73.26 — 0.86
R053207 1–3 1,500 78.35 — 0.72
R053209 1–3 1,500 85.54 — 0.81
R13 R016201 1–3 1,300 78.50 — 0.60
R016202 1–3 1,500 83.62 — 0.76
R016204 1–4 1,200 79.85 — 0.77
R016205 1–3 1,200 80.77 — 0.77
R016207 1–3 1,200 92.60 — 0.94
R016210 1–4 1,200 64.21 — 0.73
R016211 1–3 1,200 83.05 — 0.81
R016212 1–3 1,200 87.48 — 0.89
R016213 1–3 1,200 78.90 — 0.76
R14 R026501 1–3 1,500 80.66 — 0.80
R026801 1–4 1,400 75.37 — 0.83
R027001 1–2 1,300 89.82 0.67 —
R027201 1–3 1,400 86.34 — 0.84
R15 R028401 1–3 2,100 85.84 — 0.84
R028501 1–3 2,000 86.74 — 0.86
R028801 1–4 1,600 75.78 — 0.86
R029601 1–3 1,400 81.55 — 0.79
Last updated 23 June 2010 (GF)

Printer-friendly Version


Back to Top