- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Main Assessment, by Item and Block: 2009
| Block | Item | Range of response codes |
Sample size | Percent agreement | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|---|
| — Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Mathematics Assessment. |
||||||
| M3 | M149901 | 1–3 | 1,600 | 98 | — | 0.98 |
| M150801 | 1–3 | 1,500 | 98 | — | 0.98 | |
| M150901 | 1–2 | 1,600 | 100 | 0.99 | — | |
| M151201 | 1–2 | 1,500 | 100 | 0.99 | — | |
| M4 | M151501 | 1–3 | 1,500 | 95 | — | 0.96 |
| M151601 | 1–3 | 1,400 | 96 | — | 0.95 | |
| M152602 | 1–5 | 1,500 | 99 | — | 0.99 | |
| M5 | M019701 | 1–2 | 1,600 | 100 | 0.98 | — |
| M020001 | 1–2 | 1,500 | 100 | 0.99 | — | |
| M047901 | 1–3 | 1,400 | 100 | — | 0.99 | |
| M053101 | 1–5 | 1,200 | 91 | — | 0.95 | |
| M066301 | 1–3 | 1,500 | 98 | — | 0.96 | |
| M066501 | 1–3 | 1,500 | 97 | — | 0.97 | |
| M067901 | 1–3 | 1,400 | 99 | — | 0.98 | |
| M085701 | 1–3 | 1,600 | 99 | — | 0.99 | |
| M085901 | 1–3 | 1,500 | 98 | — | 0.98 | |
| M6 | M141301 | 1–4 | 1,500 | 98 | — | 0.97 |
| M141601 | 1–2 | 1,500 | 88 | 0.74 | — | |
| M141901 | 1–5 | 1,400 | 90 | — | 0.94 | |
| M7 | M165301 | 1–3 | 1,500 | 98 | — | 0.98 |
| M167001 | 1–2 | 1,400 | 94 | 0.69 | — | |
| M168501 | 1–2 | 1,300 | 99 | 0.98 | — | |
| M168502 | 1–2 | 1,300 | 99 | 0.98 | — | |
| M168503 | 1–3 | 1,100 | 96 | — | 0.91 | |
| M168701 | 1–3 | 1,600 | 98 | — | 0.99 | |
| M170101 | 1–3 | 1,500 | 100 | — | 1.00 | |
| M8 | M164601 | 1–3 | 1,500 | 100 | — | 0.99 |
| M166301 | 1–3 | 1,600 | 96 | — | 0.97 | |
| M169901 | 1–2 | 1,500 | 99 | 0.97 | — | |
| M169902 | 1–2 | 1,500 | 100 | 0.99 | — | |
| M169903 | 1–3 | 1,400 | 97 | — | 0.97 | |
| M169904 | 1–2 | 1,300 | 98 | 0.92 | — | |
| M170501 | 1–3 | 1,500 | 95 | — | 0.94 | |
| M9 | M165001 | 1–2 | 1,400 | 99 | 0.96 | — |
| M165501 | 1–3 | 1,600 | 99 | — | 0.99 | |
| M165701 | 1–3 | 1,500 | 99 | — | 0.99 | |
| M165702 | 1–3 | 1,400 | 96 | — | 0.97 | |
| M168901 | 1–2 | 1,600 | 99 | 0.98 | — | |
| M170801 | 1–3 | 1,500 | 99 | — | 0.95 | |
| M10 | M142801 | 1–2 | 1,500 | 100 | 1.00 | — |
| M143201 | 1–2 | 1,500 | 100 | 1.00 | — | |
| M143501 | 1–5 | 1,500 | 94 | — | 0.95 | |
| M11 | M104901 | 1–3 | 1,400 | 99 | — | 0.98 |
| M162901 | 1–3 | 1,600 | 97 | — | 0.97 | |
| M163101 | 1–3 | 1,600 | 96 | — | 0.97 | |
| M167901 | 1–3 | 1,500 | 98 | — | 0.97 | |
| M168301 | 1–3 | 1,400 | 97 | — | 0.97 | |
| M12 | M153201 | 1–3 | 1,600 | 96 | — | 0.95 |
| M153601 | 1–2 | 1,600 | 100 | 0.99 | — | |
| M153901 | 1–3 | 1,500 | 92 | — | 0.94 | |
Last updated 06 January 2016 (GF)
