Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2005, by Block and Item, Grade 8 Science Combined National and State Main Assessment: 2005
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2005, by block and item, grade 8 science combined national and state main assessment: 2005
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa Intraclass correlation
B7 K097901 1–3 200 90 0.78
K098501 1–3 200 93 0.91
K101201 1–3 200 92 0.55
K101801 1–3 800 80 0.74
K102001 1–4 200 76 0.74
K102301 1–3 200 97 0.97
B8 K041306 1–3 200 78 0.81
K041307 1–3 200 84 0.80
K041401 1–3 200 85 0.82
K041402 1–3 200 93 0.89
K041403 1–3 200 84 0.75
B9 K031602 1–3 900 93 0.94
K031603 1–3 900 99 0.97
K031604 1–3 900 98 0.96
K031606 1–3 900 89 0.88
K031607 1–4 900 87 0.90
K031608 1–3 900 87 0.80
K031609 1–3 900 93 0.93
K031610 1–3 900 97 0.94
K031611 1–3 800 98 0.94
K031613 1–3 900 98 0.94
B10 K092601 1–3 100 92 0.87
K093601 1–3 500 94 0.90
K095801 1–2 100 95 0.87
K095901 1–3 100 95 0.94
K096101 1–3 100 87 0.83
K099001 1–3 100 88 0.78
B11 K043001 1–3 700 92 0.80
K043101 1–3 700 84 0.84
K043102 1–4 600 77 0.87
K043103 1–3 700 92 0.79
K043501 1–3 700 86 0.83
K043601 1–3 600 94 0.93
K043602 1–3 500 97 0.88
K043603 1–3 600 91 0.91
B12 K047201 1–4 100 84 0.93
K047301 1–3 100 92 0.89
K047401 1–3 100 91 0.91
K047901 1–3 100 84 0.84
K048001 1–3 100 98 0.97
K048101 1–4 600 91 0.88
K048102 1–3 500 97 0.79
K048103 1–2 600 91 0.80
B13 K048601 1–3 100 92 0.91
K048901 1–3 100 98 0.98
K049001 1–3 700 99 0.87
K049301 1–3 100 98 0.73
K049401 1–3 100 88 0.70
K049402 1–3 100 78 0.82
K049403 1–3 100 91 0.79
K049404 1–4 100 84 0.81
B14 K035601 1–3 200 84 0.83
K035801 1–3 200 90 0.91
K035901 1–3 200 92 0.88
K036101 1–3 200 83 0.79
K036301 1–3 200 91 0.94
K036401 1–3 200 94 0.91
K036402 1–3 200 90 0.86
K036403 1–3 200 89 0.81
K036404 1–3 200 88 0.47
K036701 1–3 800 97 0.92
K036801 1–3 200 93 0.88
B15 K037301 1–3 100 99 0.99
K037401 1–3 100 84 0.59
K037501 1–3 600 96 0.91
K037601 1–3 100 88 0.90
K037701 1–4 100 100 1.00
K037703 1–3 100 92 0.87
K038101 1–2 100 99 0.96
K038201 1–3 100 94 0.82
K038301 1–5 100 89 0.91
B20 K092801 1–3 200 92 0.87
K093701 1–3 100 91 0.88
K093901 1–3 200 85 0.87
K094901 1–3 100 87 0.58
K095401 1–3 100 93 0.82
K097001 1–3 100 91 0.85
B21 K045301 1–3 700 84 0.83
K045601 1–4 700 96 0.94
K045701 1–3 700 81 0.83
K045801 1–3 700 89 0.85
K046301 1–3 700 95 0.93
K046401 1–3 600 94 0.87
K046501 1–3 600 87 0.83
K046601 1–3 600 84 0.82
K046701 1–4 600 88 0.88
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2005 Science Assessment.

Last updated 09 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education