- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 9 Mathematics Long-Term Trend Assessment: 2004
| Block | Item | Range of response codes | Sample size | Percent exact agreement | Cohen's Kappa |
|---|---|---|---|---|---|
| M21 | N292101 | 1–2 | 900 | 100 | 1.00 |
| N292401 | 1–2 | 900 | 100 | 1.00 | |
| N293001 | 1–2 | 900 | 100 | 1.00 | |
| N293301 | 1–2 | 900 | 100 | 0.99 | |
| N293501 | 1–2 | 900 | 100 | 1.00 | |
| M22 | N294201 | 1–2 | 900 | 100 | 0.98 |
| N294301 | 1–2 | 900 | 99 | 1.00 | |
| N295301 | 1–2 | 900 | 100 | 0.99 | |
| N295401 | 1–2 | 900 | 99 | 0.98 | |
| N295701 | 1–2 | 800 | 100 | 0.99 | |
| M23 | N310401 | 1–2 | 900 | 100 | 0.95 |
| N310701 | 1–2 | 900 | 100 | 0.96 | |
| N310702 | 1–2 | 900 | 100 | 0.98 | |
| N310703 | 1–2 | 900 | 99 | 0.98 | |
| N311101 | 1–2 | 900 | 100 | 1.00 | |
| N311201 | 1–2 | 900 | 100 | 1.00 | |
| N311301 | 1–2 | 900 | 100 | 0.99 | |
| N311501 | 1–2 | 900 | 100 | 0.99 | |
| N311502 | 1–2 | 800 | 99 | 0.98 | |
| M24 | N312901 | 1–2 | 900 | 99 | 0.98 |
| N313001 | 1–2 | 900 | 100 | 0.99 | |
| N313101 | 1–2 | 900 | 99 | 0.98 | |
| N313102 | 1–2 | 900 | 99 | 0.98 | |
| N313103 | 1–2 | 900 | 99 | 0.99 | |
| N313701 | 1–2 | 800 | 100 | 0.99 | |
| N313702 | 1–2 | 700 | 99 | 0.99 | |
| N313801 | 1–2 | 800 | 100 | 0.99 | |
| N313901 | 1–2 | 800 | 100 | 0.99 | |
| MX51 | N287601 | 1–2 | 800 | 99 | 0.98 |
| N288101 | 1–2 | 900 | 100 | 0.99 | |
| N288501 | 1–2 | 800 | 100 | 0.99 | |
| N289201 | 1–2 | 800 | 100 | 1.00 | |
| MX52 | N289701 | 1–2 | 900 | 100 | 1.00 |
| N289901 | 1–2 | 900 | 100 | 0.98 | |
| N290101 | 1–2 | 900 | 100 | 0.99 | |
| N290701 | 1–2 | 900 | 100 | 0.98 | |
| N291501 | 1–2 | 800 | 100 | 1.00 | |
| NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment. |
|||||
Last updated 06 April 2009 (GF)
