Skip Navigation
small NCES header image

Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 13 Mathematics Long-Term Trend Assessment: 2004
Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 13 mathematics long-term trend assessment: 2004
Block Item Range of response codes Sample size Percent exact agreement Cohen's Kappa
M21 N292401 1–2 1,000 100 0.99
N300201 1–2 900 100 0.99
N300401 1–2 900 99 0.99
N300501 1–2 900 99 0.98
N301501 1–2 900 100 0.98
M22 N301801 1–2 1,000 100 1.00
N302401 1–2 900 100 0.99
N302601 1–2 800 100 0.99
N302901 1–2 900 100 0.98
N303201 1–2 900 100 0.99
M23 N310701 1–2 900 100 0.98
N310702 1–2 1,000 100 0.99
N310703 1–2 1,000 99 0.97
N313101 1–2 900 100 0.99
N313102 1–2 1,000 100 0.99
N313103 1–2 900 100 0.99
N315101 1–2 900 100 0.99
N315501 1–2 900 100 0.99
N315701 1–2 900 99 0.98
M24 N316601 1–2 900 100 0.99
N316602 1–2 900 100 0.94
N316603 1–2 900 100 0.98
N316701 1–2 900 100 0.98
N317101 1–2 900 100 0.99
N317102 1–2 900 99 0.98
N317103 1–2 900 100 0.99
N318801 1–2 800 98 0.95
MX51 N288101 1–2 900 100 1.00
N296401 1–2 900 99 0.99
N296701 1–2 900 99 0.98
N296901 1–2 900 99 0.98
N297001 1–2 900 100 1.00
MX52 N289901 1–2 900 99 0.93
N290101 1–2 900 99 0.99
N298201 1–2 900 100 1.00
N298601 1–2 900 100 0.98
N299001 1–2 900 100 0.99
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version


Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education