Skip to main content

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2014, Rescored in 2018, Grade 8 U.S. History National Assessment, by Block and Item: 2018

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2014, Rescored in 2018, Grade 8 U.S. History National Assessment, by Block and Item: 2018

Score range and percentage of exact agreement for the constructed-response items from previous year 2014, rescored in 2018, grade 8 U.S. history PBA national assessment, by block and item: 2018
BlockItemScore rangeSample sizePercentage of exact agreement
H3H0358011-380087
H0359011-380092
H0359021-380085
H0361011-380088
H0364021-480076
H4H0590011-380087
H0592011-380077
H0597011-380090
H0598011-380088
H0602011-380085
H5H0381031-480084
H0383011-380088
H0386011-380085
H0387021-380083
H0390011-380098
H6H0690011-380071
H0694011-380085
H0696011-380088
H0698011-480079
H0700011-280092
H7H0823011-380086
H0843011-480082
H0849011-380086
H0902011-480084
H8H0829011-380070
H0884011-480077
H0911011-480083
H0978011-480072
H9H0824021-380094
H0848011-380084
H0874011-480081
H0883011-380089
H0979011-380099
H10H0422011-380093
H0428011-480084
H0429021-380087
H0430011-380076
H0431011-380086
H11H0704011-380084
H0707011-480096
H0708011-380088
H0710011-480081
H0716011-380080
H12H0719011-380081
H0726011-380082
H0727011-480089
H0730011-380085
H0731011-380078
NOTE: PBA = Paper-based assessment. Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure. The block naming conventions used in this table are described in the document 2018 Block Naming Conventions in Data Products and TDW PDF File .
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2018 U.S. History Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2014, rescored in 2018, grade 8 U.S. history PBA national assessment, by block and item: 2018
BlockItemScore rangeSample sizeCohen’s KappaIntraclass correlation
H3H0358011-3700.65.80
H0359011-3800.87.87
H0359021-3700.70.82
H0361011-3700.76.86
H0364021-4800.63.81
H4H0590011-3800.80.90
H0592011-3700.63.77
H0597011-3700.80.90
H0598011-3700.79.88
H0602011-3700.74.83
H5H0381031-4700.55.66
H0383011-3800.79.88
H0386011-3800.64.71
H0387021-3700.62.70
H0390011-3700.95.98
H6H0690011-3800.52.70
H0694011-3700.71.84
H0696011-3800.79.80
H0698011-4700.66.82
H0700011-2700.82†
H7H0823011-3700.76.86
H0843011-4800.71.85
H0849011-3800.76.87
H0902011-4700.69.87
H8H0829011-3700.50.67
H0884011-4800.65.85
H0911011-4700.75.91
H0978011-4800.57.81
H9H0824021-3800.86.90
H0848011-3700.61.67
H0874011-4700.66.77
H0883011-3700.74.81
H0979011-3800.99.99
H10H0422011-3700.79.85
H0428011-4800.70.82
H0429021-3700.78.87
H0430011-3800.51.64
H0431011-3700.73.80
H11H0704011-3800.71.83
H0707011-4700.88.96
H0708011-3700.77.84
H0710011-4800.62.72
H0716011-3700.63.75
H12H0719011-3700.68.83
H0726011-3700.50.55
H0727011-4800.82.92
H0730011-3700.75.86
H0731011-3700.64.79
† Not applicable. The intraclass correlation is not reported for dichotomously scored items.
NOTE: PBA = Paper-based assessment. The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation. The block naming conventions used in this table are described in the document 2018 Block Naming Conventions in Data Products and TDW PDF File.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2018 U.S. History Assessment.


Last updated 17 January 2023 (ML)