Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2012, Rescored in 2020, Age 9 Reading Long-Term Trend National Assessment, by Block and Item: 2020
NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2012, Rescored in 2020, Age 9 Reading Long-Term Trend National Assessment, by Block and Item: 2020
| Block | Item | Score range | Sample size | Percentage of exact agreement |
|---|---|---|---|---|
| Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure. | ||||
| U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2020 Reading Long-Term Trend Assessment. | ||||
| R1 | N021203 | 5 | 1,000 | 77 |
| R3 | N021803 | 4 | 1,000 | 93 |
| R4 | N022103 | 4 | 1,000 | 83 |
| Block | Item | Score range | Sample size | Cohen's Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation. | |||||
| U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2020 Reading Long-Term Trend Assessment. | |||||
| R1 | N021203 | 5 | 900 | .66 | .85 |
| R3 | N021803 | 4 | 900 | .86 | .92 |
| R4 | N022103 | 4 | 900 | .69 | .77 |
Last updated 28 March 2024 (SK)