Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percentage Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From 1996 That Were Rescored in 2000, by Block and Item, Grade 4 Science Assessment: 2000
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from 1996 that were rescored in 2000, by block and item, grade 4 science assessment: 2000
Block Item Range of response codes Sample size Percentage exact agreement Cohen's Kappa Intraclass correlation
S3 K031001 1–3 300 95 0.89
K031002 1–3 300 83 0.78
K031003 1–3 300 84 0.73
K031004 1–2 300 93 0.81
K031005 1–3 300 82 0.76
K031006 1–3 300 91 0.92
K031007 1–3 300 84 0.86
S4 K031101 1–2 400 97 0.90
K031102 1–2 400 94 0.87
K031103 1–2 400 90 0.79
K031104 1–2 400 98 0.84
K031105 1–3 400 99 0.99
K031107 1–4 400 90 0.94
S6 K031301 1–4 200 91 0.72
K031302 1–3 200 96 0.80
K031303 1–3 200 92 0.92
K031304 1–3 200 92 0.92
K031309 1–4 200 94 0.94
S7 K031401 1–4 600 78 0.54
K031402 1–3 600 86 0.59
K031403 1–3 600 84 0.57
K031404 1–3 400 83 0.53
K031407 1–3 600 75 0.46
K031408 1–3 600 87 0.56
K031409 1–3 600 85 0.63
K031410 1–3 600 86 0.62
S9 K031602 1–3 600 89 0.81
K031603 1–3 600 92 0.64
K031604 1–3 600 92 0.82
K031606 1–3 600 80 0.48
K031607 1–4 600 78 0.70
K031608 1–3 600 83 0.55
K031609 1–3 600 87 0.65
S10 K031901 1–3 600 77 0.63
K032001 1–3 600 87 0.74
K032501 1–3 600 83 0.60
K032502 1–3 600 87 0.53
K032601 1–3 600 84 0.68
K032602 1–3 600 84 0.64
S12 K034001 1–3 600 85 0.62
K034101 1–3 600 83 0.72
KW34101 1–3 600 89 0.91
KX34101 1–3 600 88 0.67
KY34101 1–3 600 75 0.67
KZ34101 1–3 600 82 0.53
KZ34401 1–3 600 82 0.67
KZ34501 1–4 600 81 0.64
KZ34502 1–3 600 89 0.81
S13 K034802 1–3 600 94 0.90
K034901 1–3 600 86 0.54
K034902 1–3 600 87 0.74
K035201 1–3 600 80 0.73
K035301 1–4 600 85 0.53
S14 K035601 1–3 600 81 0.56
K035801 1–3 600 85 0.83
K035901 1–3 600 83 0.60
K036101 1–3 600 82 0.64
K036301 1–3 600 86 0.76
S15 K037301 1–3 600 87 0.67
K037401 1–3 600 81 0.24
K037501 1–3 600 91 0.26
K037601 1–3 600 79 0.58
K037701 1–4 600 91 0.56
K037702 1–3 600 83 0.58
S21 K039801 1–3 600 87 0.54
K039901 1–5 400 78 0.69
K040001 1–3 600 83 0.69
K040301 1–3 600 80 0.64
K040401 1–4 600 80 0.76
K040501 1–3 600 84 0.64
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment.

Last updated 15 July 2008 (KL)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education