Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Grade 4 Science Combined National and State Main Assessment: 2005
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, grade 4 science combined national and state main assessment: 2005
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa Intraclass correlation
B3 K031001 1–3 1,300 98 0.96
K031002 1–3 1,300 95 0.94
K031003 1–3 1,300 94 0.90
K031004 1–2 1,200 95 0.90
K031005 1–3 1,200 92 0.88
K031006 1–3 1,100 95 0.96
K031007 1–3 1,000 92 0.93
B4 K031101 1–2 1,300 98 0.92
K031102 1–2 1,300 96 0.92
K031103 1–2 1,300 94 0.89
K031104 1–2 1,200 98 0.93
K031105 1–3 1,200 98 0.96
K031107 1–4 1,100 95 0.96
B6 K031301 1–4 1,200 93 0.91
K031302 1–3 1,000 90 0.45
K031303 1–3 900 87 0.89
K031304 1–3 900 91 0.91
K031309 1–4 1,100 92 0.93
B7 K031401 1–4 1,400 87 0.85
K031402 1–3 1,400 88 0.88
K031403 1–3 1,400 88 0.89
K031404 1–3 1,200 94 0.94
K031407 1–3 1,400 90 0.89
K031408 1–3 1,200 95 0.94
K031409 1–3 1,400 91 0.86
K031410 1–3 1,400 93 0.93
B8 K103101 1–3 1,400 92 0.86
K103901 1–3 1,300 92 0.87
B10 K031901 1–3 1,400 90 0.90
K032001 1–3 1,400 91 0.91
K032501 1–3 1,400 92 0.94
K032502 1–3 1,400 92 0.91
K032601 1–3 1,400 91 0.87
K032602 1–3 1,300 92 0.89
B11 K092201 1–3 1,300 92 0.90
K098201 1–3 1,400 90 0.84
K098301 1–3 1,400 88 0.79
K099501 1–3 1,400 94 0.93
B13 K034802 1–3 1,400 89 0.84
K034901 1–3 1,400 96 0.95
K034902 1–3 1,400 94 0.90
K035201 1–3 1,400 92 0.93
K035301 1–4 1,400 93 0.91
B14 K035601 1–3 1,400 93 0.88
K035801 1–3 1,400 92 0.94
K035901 1–3 1,300 98 0.94
K036101 1–3 1,300 91 0.88
K036301 1–3 1,300 98 0.96
B15 K037301 1–3 1,400 97 0.95
K037401 1–3 1,100 95 0.73
K037601 1–3 1,300 90 0.83
K037701 1–4 1,400 98 0.97
K037702 1–3 1,300 89 0.84
B20 K096901 1–3 1,500 93 0.94
K098401 1–3 1,400 96 0.93
K099601 1–3 1,400 93 0.92
K100701 1–3 1,400 96 0.97
S9 K060203 1–3 1,300 88 0.77
K060204 1–3 1,500 93 0.93
K060206 1–3 1,500 90 0.87
K060207 1–4 1,400 86 0.92
K060301 1–3 1,400 93 0.89
S10 K060401 1–4 1,300 90 0.91
S12 K060901 1–3 1,300 86 0.84
S14 K061601 1–3 1,300 97 0.97
S15 K062001 1–4 1,400 88 0.86
K062201 1–4 1,500 91 0.94
K062401 1–3 1,400 92 0.93
K062903 1–4 1,300 95 0.96
K063001 1–3 1,100 94 0.89
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2005 Science Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education