- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 12 Mathematics Combined National and State Assessment, by Item and Block: 2013
| Block | Item | Range of response codes | Sample size | Percent exact agreement |
|---|---|---|---|---|
| NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment. | ||||
| M1 | M175901 | 1-3 | 6,600 | .96 |
| M1 | M178001 | 1-3 | 6,600 | .96 |
| M1 | M183601 | 1-3 | 6,600 | .98 |
| M1 | M183901 | 1-2 | 6,600 | .98 |
| M2 | M302901 | 1-3 | 6,500 | .98 |
| M2 | M306701 | 1-3 | 6,500 | .89 |
| M2 | M307501 | 1-3 | 6,500 | .91 |
| M3 | M179201 | 1-3 | 6,500 | .97 |
| M3 | M193401 | 1-3 | 6,500 | .93 |
| M3 | M197303 | 1-2 | 6,500 | .99 |
| M3 | M197304 | 1-2 | 6,500 | .99 |
| M4 | M174801 | 1-3 | 6,500 | .98 |
| M4 | M184201 | 1-2 | 6,500 | 1.00 |
| M4 | M184207 | 1-3 | 6,500 | .89 |
| M4 | M197601 | 1-3 | 6,500 | .94 |
| M5 | M177301 | 1-3 | 6,500 | .89 |
| M5 | M185201 | 1-2 | 6,500 | 100 |
| M5 | M185703 | 1-5 | 6,500 | .98 |
| M5 | M187701 | 1-3 | 6,500 | .99 |
| M6 | M175002 | 1-5 | 6,500 | .98 |
| M6 | M175601 | 1-3 | 6,500 | .95 |
| M6 | M177202 | 1-2 | 6,500 | .96 |
| M6 | M183401 | 1-3 | 6,500 | .97 |
| M7 | M304001 | 1-3 | 6,700 | .98 |
| M7 | M310001 | 1-4 | 6,700 | .95 |
| M7 | M310401 | 1-2 | 6,700 | 1.00 |
| M8 | M188001 | 1-3 | 7,000 | .95 |
| M8 | M188201 | 1-5 | 7,000 | .98 |
| M8 | M189001 | 1-3 | 7,000 | .93 |
| M8 | M193101 | 1-2 | 7,000 | 1.00 |
| M9 | M191901 | 1-5 | 6,700 | .95 |
| M9 | M198701 | 1-2 | 6,700 | 1.00 |
| M9 | M198901 | 1-3 | 6,700 | .96 |
| M10 | M178501 | 1-3 | 7,000 | .98 |
| M10 | M190801 | 1-3 | 7,000 | .98 |
| M10 | M191701 | 1-3 | 7,000 | .99 |
| M10 | M199201 | 1-3 | 7,000 | .99 |
| M11 | M181201 | 1-2 | 6,600 | .99 |
| M11 | M181202 | 1-3 | 6,600 | .99 |
| M11 | M181203 | 1-2 | 6,600 | 1.00 |
| M11 | M183101 | 1-2 | 6,600 | .99 |
| M12 | M178701 | 1-4 | 6,600 | .98 |
| M12 | M181801 | 1-2 | 6,600 | .99 |
| M12 | M186001 | 1-2 | 6,600 | 1.00 |
| M12 | M194001 | 1-3 | 6,600 | .97 |
| M13 | M302101 | 1-4 | 6,500 | .98 |
| M13 | M302701 | 1-3 | 6,500 | 1.00 |
| M13 | M305501 | 1-2 | 6,500 | .99 |
| M13 | M306501 | 1-2 | 6,500 | 1.00 |
| M14 | M308001 | 1-2 | 6,700 | 1.00 |
| M14 | M310501 | 1-3 | 6,700 | .99 |
| Block | Item | Range of response codes | Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
† The intraclass correlation is not reported for dichotomously scored items. | |||||
| M1 | M175901 | 1-3 | 5,500 | .77 | .80 |
| M1 | M178001 | 1-3 | 5,200 | .90 | .94 |
| M1 | M183601 | 1-3 | 6,300 | .97 | .98 |
| M1 | M183901 | 1-2 | 6,200 | .98 | † |
| M2 | M302901 | 1-3 | 6,100 | .97 | .98 |
| M2 | M306701 | 1-3 | 6,400 | .83 | .90 |
| M2 | M307501 | 1-3 | 6,300 | .80 | .87 |
| M3 | M179201 | 1-3 | 4,400 | .87 | .90 |
| M3 | M193401 | 1-3 | 6,400 | .81 | .86 |
| M3 | M197303 | 1-2 | 5,800 | .98 | † |
| M3 | M197304 | 1-2 | 5,800 | 1.00 | † |
| M4 | M174801 | 1-3 | 4,800 | .82 | .83 |
| M4 | M184201 | 1-2 | 6,200 | 1.00 | † |
| M4 | M184207 | 1-3 | 5,200 | .69 | .82 |
| M4 | M197601 | 1-3 | 5,500 | .89 | .94 |
| M5 | M177301 | 1-3 | 6,400 | .81 | .91 |
| M5 | M185201 | 1-2 | 6,200 | .99 | † |
| M5 | M185703 | 1-5 | 5,900 | .96 | .97 |
| M5 | M187701 | 1-3 | 6,000 | .98 | .99 |
| M6 | M175002 | 1-5 | 6,400 | .97 | .99 |
| M6 | M175601 | 1-3 | 6,500 | .90 | .94 |
| M6 | M177202 | 1-2 | 5,400 | .91 | † |
| M6 | M183401 | 1-3 | 4,000 | .87 | .95 |
| M7 | M304001 | 1-3 | 6,600 | .96 | .97 |
| M7 | M310001 | 1-4 | 6,200 | .93 | .98 |
| M7 | M310401 | 1-2 | 6,600 | .99 | † |
| M8 | M188001 | 1-3 | 6,400 | .92 | .97 |
| M8 | M188201 | 1-5 | 6,400 | .92 | .97 |
| M8 | M189001 | 1-3 | 6,100 | .85 | .85 |
| M8 | M193101 | 1-2 | 6,900 | .99 | † |
| M9 | M191901 | 1-5 | 6,100 | .90 | .98 |
| M9 | M198701 | 1-2 | 6,600 | .99 | † |
| M9 | M198901 | 1-3 | 5,800 | .94 | .96 |
| M10 | M178501 | 1-3 | 6,400 | .98 | .99 |
| M10 | M190801 | 1-3 | 6,400 | .97 | .98 |
| M10 | M191701 | 1-3 | 6,800 | .98 | .99 |
| M10 | M199201 | 1-3 | 6,500 | .98 | .98 |
| M11 | M181201 | 1-2 | 6,400 | .99 | † |
| M11 | M181202 | 1-3 | 6,400 | .98 | .98 |
| M11 | M181203 | 1-2 | 6,200 | 1.00 | † |
| M11 | M183101 | 1-2 | 4,900 | .99 | † |
| M12 | M178701 | 1-4 | 4,300 | .94 | .95 |
| M12 | M181801 | 1-2 | 5,500 | .99 | † |
| M12 | M186001 | 1-2 | 6,300 | 1.00 | † |
| M12 | M194001 | 1-3 | 5,700 | .95 | .97 |
| M13 | M302101 | 1-4 | 5,600 | .93 | .97 |
| M13 | M302701 | 1-3 | 6,400 | 1.00 | 1.00 |
| M13 | M305501 | 1-2 | 6,400 | .99 | † |
| M13 | M306501 | 1-2 | 6,300 | .99 | † |
| M14 | M308001 | 1-2 | 6,600 | 1.00 | † |
| M14 | M310501 | 1-3 | 6,500 | .99 | 1.00 |
Last updated 21 May 2017 (DC)
