Page Title:
Keywords:
Description:
Skip to main content
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 12 Reading National Main Assessment, by Item and Block: 2009

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2009, grade 12 reading national main assessment, by item and block: 2009
Block Item Range of response codes Sample size Percent agreement Cohen’s Kappa Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Reading Assessment.
F3 R017101 1–2 600 91 0.75 —
R017102 1–3 600 84 — 0.85
R017104 1–3 600 83 — 0.83
R017105 1–4 600 66 — 0.71
R017107 1–3 600 85 — 0.85
R017108 1–2 600 93 0.86 —
R017110 1–3 600 91 — 0.92
F4 R013501 1–2 600 90 0.78 —
R013503 1–2 600 90 0.80 —
R013505 1–2 600 88 0.73 —
R013506 1–4 600 68 — 0.73
R013508 1–2 600 88 0.75 —
R013509 1–2 600 91 0.79 —
F5 R016301 1–3 600 81 — 0.78
R016302 1–3 600 78 — 0.74
R016303 1–3 600 85 — 0.81
R016305 1–3 600 76 — 0.75
R016306 1–3 600 82 — 0.85
R016307 1–3 500 77 — 0.72
R016308 1–4 500 78 — 0.82
F6 R013201 1–4 600 77 — 0.86
R013203 1–2 600 97 0.64 —
R013205 1–2 600 97 0.65 —
R013207 1–2 600 88 0.64 —
R013209 1–2 600 93 0.86 —
R013211 1–2 600 88 0.76 —
R013212 1–4 600 72 — 0.66
F7 R013701 1–2 600 86 0.71 —
R013702 1–2 600 88 0.74 —
R013704 1–2 600 90 0.66 —
R013706 1–4 500 73 — 0.68
R013708 1–2 500 86 0.72 —
R013710 1–2 500 87 0.70 —
R013712 1–2 500 76 0.53 —
F8 R053501 1–3 600 68 — 0.58
R053504 1–2 700 96 0.84 —
R053507 1–4 600 85 — 0.88
R053509 1–2 600 92 0.74 —
F9 R053301 1–3 600 93 — 0.83
R053303 1–2 600 88 0.70 —
R053305 1–3 600 85 — 0.80
R053309 1–3 600 87 — 0.90
F10 R013402 1–2 600 98 0.94 —
R013403 1–4 600 92 — 0.95
R013405 1–2 600 94 0.81 —
R013406 1–4 500 76 — 0.90
R013407 1–2 600 87 0.70 —
R013409 1–2 600 95 0.79 —
R013411 1–2 600 90 0.69 —
R013413 1–2 500 80 0.57 —
F11 R019301 1–4 600 93 — 0.95
R019401 1–3 600 91 — 0.92
R019501 1–2 600 86 0.70 —
R019701 1–3 600 88 — 0.88
R020001 1–3 600 82 — 0.77
R020201 1–3 600 84 — 0.80

Last updated 07 January 2016 (GF)