Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Main Assessment, by Item and Block: 2007
| Block | Item | Range of response codes | Sample size | Percent agreement | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|---|
| — Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Reading Assessment. |
||||||
| R3 | R053901 | 1–3 | 1,900 | 86.60 | — | 0.83 |
| R054001 | 1–3 | 1,800 | 84.36 | — | 0.76 | |
| R054201 | 1–4 | 1,800 | 76.79 | — | 0.85 | |
| R054301 | 1–3 | 1,600 | 87.35 | — | 0.81 | |
| R4 | R052904 | 1–3 | 1,800 | 89.35 | — | 0.90 |
| R052906 | 1–2 | 1,800 | 90.92 | 0.81 | — | |
| R052907 | 1–4 | 1,700 | 78.74 | — | 0.82 | |
| R052909 | 1–2 | 1,600 | 94.95 | 0.87 | — | |
| R5 | R012601 | 1–2 | 1,800 | 92.60 | 0.83 | — |
| R012604 | 1–2 | 1,800 | 95.78 | 0.87 | — | |
| R012607 | 1–4 | 1,600 | 86.29 | — | 0.84 | |
| R012612 | 1–2 | 1,400 | 95.49 | 0.91 | — | |
| R6 | R017301 | 1–2 | 1,900 | 95.24 | 0.86 | — |
| R017303 | 1–3 | 1,800 | 92.48 | — | 0.93 | |
| R017307 | 1–4 | 1,600 | 86.87 | — | 0.87 | |
| R017309 | 1–3 | 1,400 | 85.76 | — | 0.86 | |
| R017310 | 1–3 | 1,700 | 96.52 | — | 0.97 | |
| R7 | R034601 | 1–3 | 1,800 | 91.27 | — | 0.91 |
| R034901 | 1–3 | 1,700 | 91.85 | — | 0.91 | |
| R035001 | 1–3 | 1,700 | 89.16 | — | 0.91 | |
| R035201 | 1–3 | 1,600 | 95.03 | — | 0.95 | |
| R035301 | 1–4 | 1,600 | 84.68 | — | 0.90 | |
| R8 | R054801 | 1–3 | 1,800 | 97.60 | — | 0.98 |
| R055101 | 1–4 | 1,700 | 91.30 | — | 0.96 | |
| R055301 | 1–3 | 1,600 | 86.88 | — | 0.85 | |
| R055401 | 1–3 | 1,500 | 94.19 | — | 0.93 | |
| R9 | R053003 | 1–2 | 1,900 | 97.62 | 0.94 | — |
| R053004 | 1–3 | 1,900 | 96.16 | — | 0.96 | |
| R053006 | 1–4 | 1,800 | 86.30 | — | 0.94 | |
| R053009 | 1–2 | 1,700 | 92.00 | 0.77 | — | |
| R053010 | 1–3 | 1,600 | 89.96 | — | 0.88 | |
| R10 | R053101 | 1–2 | 1,900 | 92.19 | 0.75 | — |
| R053105 | 1–3 | 1,800 | 86.99 | — | 0.87 | |
| R053106 | 1–4 | 1,700 | 85.69 | — | 0.93 | |
| R053108 | 1–3 | 1,500 | 91.25 | — | 0.89 | |
| R11 | R020401 | 1–3 | 1,900 | 95.52 | — | 0.94 |
| R020701 | 1–2 | 1,800 | 94.62 | 0.85 | — | |
| R021201 | 1–3 | 1,700 | 89.63 | — | 0.88 | |
| R021701 | 1–3 | 1,800 | 86.51 | — | 0.85 | |
| R12 | R023201 | 1–2 | 1,600 | 96.12 | 0.92 | — |
| R023501 | 1–3 | 1,800 | 90.21 | — | 0.92 | |
| R023601 | 1–3 | 1,800 | 91.80 | — | 0.91 | |
| R024001 | 1–3 | 1,700 | 88.95 | — | 0.89 | |
Last updated 02 March 2010 (JL)