Page Title:
Keywords:
Description:
Skip to main content
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Main Assessment, by Item and Block: 2009

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 reading combined national and state main assessment, by item and block: 2009
Block Item Range of response codes Sample size Percent agreement Cohen’s Kappa Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Reading Assessment.
F3 R053901 1–3 900 87 — 0.83
R054001 1–3 900 83 — 0.78
R054201 1–4 900 76 — 0.85
R054301 1–3 800 86 — 0.79
F4 R052904 1–3 900 92 — 0.93
R052906 1–2 900 95 0.90 —
R052907 1–4 800 80 — 0.86
R052909 1–2 800 97 0.91 —
F5 R012601 1–2 900 93 0.82 —
R012604 1–2 800 97 0.91 —
R012607 1–4 800 84 — 0.83
R012612 1–2 700 89 0.76 —
F6 R017301 1–2 900 94 0.83 —
R017303 1–3 900 91 — 0.92
R017307 1–4 800 81 — 0.85
R017309 1–3 700 90 — 0.89
R017310 1–3 900 96 — 0.96
F7 R034601 1–3 900 91 — 0.92
R034901 1–3 800 91 — 0.90
R035001 1–3 800 90 — 0.89
R035201 1–3 800 92 — 0.90
R035301 1–4 700 86 — 0.89
F8 R054801 1–3 900 97 — 0.98
R055101 1–4 800 90 — 0.95
R055301 1–3 800 83 — 0.82
R055401 1–3 700 92 — 0.92
F9 R053003 1–2 900 98 0.94 —
R053004 1–3 900 96 — 0.96
R053006 1–4 900 85 — 0.93
R053009 1–2 800 92 0.77 —
R053010 1–3 800 86 — 0.86
F10 R053101 1–2 900 91 0.67 —
R053105 1–3 900 82 — 0.83
R053106 1–4 800 87 — 0.94
R053108 1–3 700 90 — 0.88
F11 R020401 1–3 900 96 — 0.96
R020701 1–2 900 92 0.80 —
R021201 1–3 800 87 — 0.85
R021701 1–3 900 88 — 0.87
F12 R023201 1–2 800 93 0.86 —
R023501 1–3 900 91 — 0.93
R023601 1–3 900 94 — 0.94
R024001 1–3 800 89 — 0.88
R1 R057704 1–2 800 95 0.89 —
R057705 1–3 800 90 — 0.89
R057706 1–3 800 91 — 0.87
R057707 1–4 700 89 — 0.93
R057710 1–3 700 89 — 0.86
R2 R057806 1–4 800 86 — 0.87
R057807 1–2 800 96 0.88 —
R057809 1–3 700 89 — 0.83
R3 R058003 1–3 800 97 — 0.95
R058004 1–3 800 87 — 0.85
R058006 1–4 800 84 — 0.87
R058007 1–2 700 96 0.91 —
R058008 1–3 700 95 — 0.92
R4 R058204 1–3 800 91 — 0.91
R058206 1–4 700 84 — 0.87
R058207 1–3 800 91 — 0.91
R058209 1–2 700 98 0.94 —
R5 R058504 1–3 800 82 — 0.74
R058508 1–4 700 77 — 0.84
R058509 1–2 600 96 0.90 —
R6 R058602 1–3 800 81 — 0.78
R058606 1–3 700 91 — 0.88
R058608 1–4 700 88 — 0.90
R7 R058805 1–3 800 85 — 0.83
R058807 1–4 700 83 — 0.88
R058810 1–3 700 91 — 0.91
R8 R059106 1–4 700 80 — 0.81
R059108 1–3 700 93 — 0.93
R059109 1–3 800 91 — 0.90
R059110 1–2 600 94 0.88 —
R9 R059403 1–3 800 84 — 0.83
R059406 1–4 800 81 — 0.88
R059407 1–3 800 89 — 0.90
R059409 1–2 700 93 0.86 —
R10 R059504 1–3 800 85 — 0.82
R059507 1–4 700 81 — 0.83
R059509 1–3 700 89 — 0.88
R059510 1–3 600 87 — 0.86

Last updated 07 January 2016 (GF)