Skip to main content
Skip Navigation
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percentage Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Grade 12 Science Assessment: 2000
NAEP Technical DocumentationRange of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, grade 12 science assessment: 2000
Block Item Range of response codes Sample size Percentage exact agreement Cohen's Kappa Intraclass correlation
S3 K049501 1?4 600 99 ? 0.99
K049502 1?5 600 93 ? 0.96
K049503 1?3 600 92 ? 0.95
K049504 1?3 600 93 ? 0.96
K049505 1?3 600 95 ? 0.98
K049506 1?3 600 93 ? 0.92
S5 K040801 1?3 600 98 ? 0.99
K040802 1?3 600 98 ? 0.98
K040803 1?3 600 98 ? 0.97
K040804 1?3 600 92 ? 0.93
K040805 1?3 600 95 ? 0.96
K040806 1?2 600 92 0.88 ?
K040808 1?3 600 98 ? 0.99
K040809 1?3 600 97 ? 0.99
S6 K049701 1?3 700 99 ? 0.99
K049702 1?3 700 98 ? 0.99
K049703 1?3 700 97 ? 0.97
K049704 1?3 700 97 ? 0.98
K049705 1?4 700 92 ? 0.96
K049706 1?3 700 92 ? 0.96
K049707 1?5 700 92 ? 0.98
K049708 1?2 700 98 0.97 ?
S8 K041306 1?3 800 94 ? 0.96
K041307 1?3 800 93 ? 0.95
K041401 1?3 800 98 ? 0.98
K041402 1?3 800 98 ? 0.98
K041403 1?3 800 91 ? 0.90
K041404 1?3 800 97 ? 0.98
K041406 1?3 800 100 ? 1.00
S9 K049901 1?3 900 91 ? 0.94
K049902 1?3 900 89 ? 0.93
K049903 1?3 900 94 ? 0.96
K049904 1?3 900 97 ? 0.98
K049907 1?3 900 94 ? 0.97
K049908 1?3 900 97 ? 0.98
K049909 1?4 900 99 ? 1.00
K049911 1?3 900 95 ? 0.95
K049912 1?5 900 88 ? 0.96
K049914 1?3 900 98 ? 0.99
S10 K090301 1?3 600 99 ? 0.99
K090501 1?3 600 99 ? 0.98
K092101 1?3 600 99 ? 0.99
K092701 1?3 600 95 ? 0.95
K094601 1?3 600 97 ? 0.95
K098601 1?3 600 92 ? 0.89
S11 K051701 1?2 600 97 0.95 ?
K051801 1?3 600 92 ? 0.94
K052301 1?3 600 94 ? 0.94
K052401 1?4 600 92 ? 0.97
K052402 1?3 600 98 ? 0.98
K052501 1?4 600 98 ? 0.99
K052502 1?4 600 98 ? 0.99
K052503 1?3 600 98 ? 0.99
S12 K047201 1?4 600 96 ? 0.99
K047301 1?3 600 97 ? 0.98
K047401 1?3 600 95 ? 0.97
K047901 1?3 600 99 ? 0.99
K048001 1?3 600 99 ? 0.99
K048101 1?4 600 87 ? 0.92
K048102 1?3 600 95 ? 0.90
K048103 1?2 600 95 0.91 ?
S13 K048601 1?3 600 96 ? 0.96
K048901 1?3 600 99 ? 1.00
K049001 1?3 600 97 ? 0.89
K049301 1?3 700 99 ? 0.98
K049401 1?3 600 97 ? 0.96
K049402 1?3 700 93 ? 0.96
K049403 1?3 600 93 ? 0.94
K049404 1?4 600 88 ? 0.91
S14 K052901 1?4 900 99 ? 1.00
K053001 1?3 900 98 ? 0.98
K053101 1?3 900 99 ? 0.99
K053102 1?3 900 100 ? 1.00
K053601 1?5 900 96 ? 0.97
K053701 1?3 900 99 ? 0.99
K053801 1?3 900 94 ? 0.95
K053901 1?3 900 98 ? 0.97
S15 K054001 1?4 600 98 ? 0.99
K054002 1?3 600 98 ? 0.98
K054003 1?3 600 100 ? 1.00
K054004 1?2 600 100 1.00 ?
K054005 1?3 600 100 ? 1.00
K054006 1?3 600 98 ? 0.98
K054007 1?3 600 100 ? 1.00
K054008 1?4 600 96 ? 0.98
S20 K092001 1?3 600 94 ? 0.97
K096501 1?3 600 100 ? 1.00
S21 K059001 1?3 600 96 ? 0.97
K059101 1?3 600 93 ? 0.91
K059201 1?4 600 96 ? 0.98
K059301 1?4 600 99 ? 0.99
K059801 1?3 600 97 ? 0.97
K059901 1?4 600 99 ? 0.99
K060001 1?3 600 94 ? 0.97
K060101 1?4 600 94 ? 0.96
? The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Science Assessment.
Last updated 15 July 2008 (KL)

Printer-friendly Version


IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences