Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 9 Mathematics Long-Term Trend Assessment: 2004
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 9 mathematics long-term trend assessment: 2004
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa
M21 N292101 1–2 900 100 1.00
N292401 1–2 900 100 1.00
N293001 1–2 900 100 1.00
N293301 1–2 900 100 0.99
N293501 1–2 900 100 1.00
M22 N294201 1–2 900 100 0.98
N294301 1–2 900 99 1.00
N295301 1–2 900 100 0.99
N295401 1–2 900 99 0.98
N295701 1–2 800 100 0.99
M23 N310401 1–2 900 100 0.95
N310701 1–2 900 100 0.96
N310702 1–2 900 100 0.98
N310703 1–2 900 99 0.98
N311101 1–2 900 100 1.00
N311201 1–2 900 100 1.00
N311301 1–2 900 100 0.99
N311501 1–2 900 100 0.99
N311502 1–2 800 99 0.98
M24 N312901 1–2 900 99 0.98
N313001 1–2 900 100 0.99
N313101 1–2 900 99 0.98
N313102 1–2 900 99 0.98
N313103 1–2 900 99 0.99
N313701 1–2 800 100 0.99
N313702 1–2 700 99 0.99
N313801 1–2 800 100 0.99
N313901 1–2 800 100 0.99
MX51 N287601 1–2 800 99 0.98
N288101 1–2 900 100 0.99
N288501 1–2 800 100 0.99
N289201 1–2 800 100 1.00
MX52 N289701 1–2 900 100 1.00
N289901 1–2 900 100 0.98
N290101 1–2 900 100 0.99
N290701 1–2 900 100 0.98
N291501 1–2 800 100 1.00
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education