Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Main Assessment, by Item and Block: 2007
| Block | Item | Range of response codes | Sample size | Percent agreement | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|---|
| — Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007Mathematics Assessment. |
||||||
| M3 | M149901 | 1–3 | 1,500 | 96.81 | — | 0.97 |
| M150801 | 1–3 | 1,400 | 97.44 | — | 0.98 | |
| M150901 | 1–2 | 1,500 | 99.29 | 0.98 | — | |
| M151201 | 1–2 | 1,400 | 99.72 | 0.99 | — | |
| M4 | M151501 | 1–3 | 1,500 | 94.89 | — | 0.96 |
| M151601 | 1–3 | 1,400 | 94.54 | — | 0.94 | |
| M152602 | 1–5 | 1,500 | 96.84 | — | 0.98 | |
| M5 | M019701 | 1–2 | 1,600 | 99.88 | 0.99 | — |
| M020001 | 1–2 | 1,600 | 99.29 | 0.98 | — | |
| M047901 | 1–3 | 1,400 | 99.15 | — | 0.98 | |
| M053101 | 1–5 | 1,400 | 87.76 | — | 0.94 | |
| M066301 | 1–3 | 1,500 | 96.95 | — | 0.94 | |
| M066501 | 1–3 | 1,500 | 96.01 | — | 0.96 | |
| M067901 | 1–3 | 1,400 | 97.68 | — | 0.97 | |
| M085701 | 1–3 | 1,600 | 98.15 | — | 0.97 | |
| M085901 | 1–3 | 1,500 | 94.60 | — | 0.93 | |
| M6 | M141301 | 1–4 | 1,500 | 96.08 | — | 0.97 |
| M141601 | 1–2 | 1,400 | 85.58 | 0.69 | — | |
| M141901 | 1–5 | 1,400 | 87.70 | — | 0.93 | |
| M7 | M144201 | 1–2 | 1,600 | 99.42 | 0.99 | — |
| M144601 | 1–3 | 1,500 | 93.72 | — | 0.93 | |
| M144901 | 1–4 | 1,400 | 90.41 | — | 0.92 | |
| M145101 | 1–4 | 1,400 | 98.01 | — | 0.93 | |
| M8 | M020901 | 1–2 | 1,400 | 90.94 | 0.82 | — |
| M021001 | 1–2 | 1,600 | 99.05 | 0.98 | — | |
| M073601 | 1–5 | 1,400 | 84.91 | — | 0.94 | |
| M092401 | 1–3 | 1,600 | 97.52 | — | 0.96 | |
| M092601 | 1–3 | 1,500 | 97.69 | — | 0.97 | |
| M9 | M013131 | 1–2 | 1,500 | 98.62 | 0.97 | — |
| M052401 | 1–2 | 1,600 | 91.37 | 0.82 | — | |
| M072901 | 1–3 | 1,500 | 93.94 | — | 0.95 | |
| M073501 | 1–3 | 1,500 | 95.88 | — | 0.94 | |
| M075301 | 1–3 | 1,500 | 94.79 | — | 0.93 | |
| M075801 | 1–3 | 1,400 | 85.58 | — | 0.87 | |
| M093801 | 1–5 | 1,400 | 91.73 | — | 0.97 | |
| M10 | M142801 | 1–2 | 1,400 | 99.65 | 0.99 | — |
| M143201 | 1–2 | 1,500 | 98.94 | 0.98 | — | |
| M143501 | 1–5 | 1,400 | 84.09 | — | 0.92 | |
| M11 | M106301 | 1–2 | 1,600 | 99.00 | 0.98 | — |
| M106901 | 1–2 | 1,500 | 87.34 | 0.70 | — | |
| M107501 | 1–2 | 1,500 | 99.67 | 0.99 | — | |
| M12 | M153201 | 1–3 | 1,600 | 88.95 | — | 0.88 |
| M153601 | 1–2 | 1,500 | 98.59 | 0.97 | — | |
| M153901 | 1–3 | 1,400 | 86.52 | — | 0.89 | |
Last updated 02 March 2010 (JL)