Skip to main content

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 12 Science National Assessment, by Block and Item: 2019

​​NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 12 Science National Assessment, by Block and Item: 2019

Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 12 science DBA national assessment, by block and item: 2019
Block Item Score range Sample size Percentage of exact agreement
DBA = Digitally based assessment. Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Science Assessment.
S2 K129601 3 500 85
S2 K174901 3 500 84
S2 K175401 3 500 91
S2 K175402 3 500 91
S3 K126202 3 500 95
S3 K126303 5 600 94
S3 K126304 3 500 92
S3 K133301 5 600 94
S3 K139001 3 500 89
S3 K157201 3 500 89
S3 K157202 3 500 96
S4 K128403 4 500 86
S4 K133801 5 500 84
S4 K137201 4 600 97
S4 K144401 4 600 87
S4 K157901 3 500 93
S5 K216601 3 600 98
S5 K220201 3 600 88
S5 K220202 3 600 97
S5 K220203 3 600 95
S5 K220204 3 500 90
S6 K126001 5 500 86
S6 K131001 4 500 93
S6 K134503 3 500 87
S6 K135001 4 500 86
S6 K139801 3 500 88
S6 K143801 3 500 96
S7 K132701 3 500 94
S7 K135401 3 500 92
S7 K136101 4 500 87
S7 K136102 3 500 95
S7 K136103 4 500 92
S7 K158301 3 500 86
S8 K127201 5 600 82
S8 K136401 3 500 93
S8 K142401 3 500 90
S8 K158401 3 500 92
S8 K158402 3 500 91
S9 K173301 3 500 95
S9 K176601 3 500 88
S9 K177601 3 600 93
S9 K177602 3 600 92
S10 K158001 4 500 98
S10 K172501 4 600 87
S10 K172901 4 500 97
S10 K174701 3 500 88
S10 K179701 3 500 86
S11 K125802 3 500 85
S11 K125803 3 500 97
S11 K127701 4 500 88
S11 K141701 4 600 93
S11 K144202 3 500 85
S12 K128701 5 500 84
S12 K129001 3 500 91
S12 K130002 4 500 93
S12 K130301 3 500 90
S12 K133401 3 500 88
S13 K179301 3 500 92
S13 K216702 3 500 88
S13 K218601 3 600 92
S13 K219901 2 600 95
H1 K197202 3 500 97
H1 K197503 3 400 89
H1 K197601 4 400 91
H3 K199001 3 500 97
H3 K199002 3 500 91
H3 K199302 3 400 94
I1 K193501 4 500 99
I1 K193601 3 500 86
I1 K193801 3 400 86
I2 K221401 3 500 97
I2 K221701 3 500 89
I5 K196101 3 500 93
I5 K196201 3 500 90
I5 K196401 3 500 92
I5 K196402 2 500 92
I5 K196601 3 500 91
I5 K196901 3 500 88
I5 K197001 3 500 89
I5 K197102 3 500 89
​

Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 12 science PBA national assessment, by block and item: 2019
Block Item Score range Sample size Percentage of exact agreement
PBA = Paper-based assessment. Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Science Assessment.
S3 K126202 3 400 96
S3 K126303 5 400 93
S3 K126304 3 400 91
S3 K133301 5 400 94
S3 K139001 3 400 86
S3 K157201 3 400 92
S3 K157202 3 400 95
S4 K128403 4 400 86
S4 K133801 5 400 82
S4 K137201 4 400 97
S4 K144401 4 400 92
S4 K157901 3 400 95
S5 K129601 3 400 88
S5 K130301 3 400 90
S5 K131001 4 400 90
S5 K143903 3 400 94
S5 K158001 4 400 95
S6 K134503 3 400 86
S6 K135001 4 400 91
S6 K139801 3 400 82
S6 K143801 3 400 96
S6 K158201 3 400 87
S6 K158202 3 400 94
S6 K158203 3 400 90
S6 K158204 3 400 93
S7 K132701 3 400 91
S7 K135401 3 400 91
S7 K136101 4 400 90
S7 K136102 3 400 91
S7 K136103 4 400 92
S7 K158301 3 400 87
S8 K127201 5 400 84
S8 K136401 3 400 93
S8 K137102 4 400 93
S8 K137651 5 400 97
S8 K142401 3 400 94
S8 K158401 3 400 95
S8 K158402 3 400 95
S9 K173301 3 400 96
S9 K176601 3 400 98
S9 K177601 3 400 95
S9 K177602 3 400 94
S10 K172501 4 400 88
S10 K172901 4 400 99
S10 K174701 3 400 89
S10 K179701 3 400 93
S11 K125802 3 400 88
S11 K125803 3 400 99
S11 K127701 4 400 87
S11 K141701 4 400 94
S11 K144202 3 400 86
S12 K128701 5 400 82
S12 K129001 3 400 93
S12 K130001 4 400 90
S12 K130002 4 400 94
S12 K133401 3 400 89
S13 K126001 5 400 83
S13 K138501 4 400 98
S13 K140301 3 400 89
S13 K140303 3 400 90
S13 K158701 3 400 97
S13 K158702 3 400 94
​

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 12 science DBA national assessment, by block and item: 2019
Block Item Score range Sample size Cohen's Kappa Intraclass correlation
Not applicable. The intraclass correlation is not reported for dichotomously scored items.
DBA = Digitally based assessment. The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Science Assessment.
S2 K129601 3 500 .73 .84
S2 K174901 3 500 .73 .82
S2 K175401 3 400 .70 .76
S2 K175402 3 400 .73 .79
S3 K126202 3 500 .87 .92
S3 K126303 5 600 .87 .96
S3 K126304 3 500 .85 .89
S3 K133301 5 600 .81 .92
S3 K139001 3 500 .80 .84
S3 K157201 3 500 .82 .90
S3 K157202 3 500 .93 .95
S4 K128403 4 500 .72 .82
S4 K133801 5 500 .75 .89
S4 K137201 4 600 .95 .98
S4 K144401 4 600 .77 .88
S4 K157901 3 500 .88 .93
S5 K216601 3 600 .92 .94
S5 K220201 3 500 .79 .86
S5 K220202 3 600 .94 .94
S5 K220203 3 500 .92 .95
S5 K220204 3 500 .84 .92
S6 K126001 5 500 .70 .91
S6 K131001 4 500 .86 .89
S6 K134503 3 500 .79 .88
S6 K135001 4 500 .69 .85
S6 K139801 3 500 .82 .90
S6 K143801 3 500 .85 .89
S7 K132701 3 500 .79 .86
S7 K135401 3 500 .88 .93
S7 K136101 4 500 .80 .89
S7 K136102 3 500 .92 .94
S7 K136103 4 500 .85 .93
S7 K158301 3 500 .73 .82
S8 K127201 5 500 .77 .92
S8 K136401 3 500 .86 .92
S8 K142401 3 500 .85 .91
S8 K158401 3 400 .86 .88
S8 K158402 3 400 .77 .86
S9 K173301 3 500 .91 .91
S9 K176601 3 500 .78 .90
S9 K177601 3 600 .88 .94
S9 K177602 3 600 .87 .94
S10 K158001 4 500 .97 .98
S10 K172501 4 600 .82 .91
S10 K172901 4 500 .96 .99
S10 K174701 3 500 .81 .89
S10 K179701 3 500 .75 .75
S11 K125802 3 500 .68 .76
S11 K125803 3 500 .92 .96
S11 K127701 4 500 .83 .93
S11 K141701 4 600 .90 .95
S11 K144202 3 500 .66 .78
S12 K128701 5 500 .77 .90
S12 K129001 3 500 .64 .69
S12 K130002 4 500 .88 .96
S12 K130301 3 500 .83 .88
S12 K133401 3 500 .79 .86
S13 K179301 3 500 .74 .88
S13 K216702 3 500 .75 .84
S13 K218601 3 600 .85 .93
S13 K219901 2 600 .87
H1 K197202 3 400 .91 .93
H1 K197503 3 400 .83 .93
H1 K197601 4 400 .88 .93
H3 K199001 3 500 .93 .93
H3 K199002 3 500 .83 .93
H3 K199302 3 400 .90 .93
I1 K193501 4 500 .99 .93
I1 K193601 3 500 .78 .93
I1 K193801 3 400 .78 .93
I2 K221401 3 500 .91 .93
I2 K221701 3 500 .68 .93
I5 K196101 3 500 .85 .93
I5 K196201 3 500 .80 .93
I5 K196401 3 500 .84 .93
I5 K196402 2 500 .81
I5 K196601 3 500 .86 .93
I5 K196901 3 500 .74 .93
I5 K197001 3 500 .82 .93
I5 K197102 3 400 .74 .93
​
​
Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 12 science PBA national assessment, by block and item: 2019
BlockItemScore rangeSample sizeCohen's KappaIntraclass correlation
PBA = Paper-based assessment. The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Science Assessment.
S3K1262023400.89.95
S3K1263035400.87.96
S3K1263043400.82.86
S3K1333015400.82.93
S3K1390013300.69.75
S3K1572013400.86.93
S3K1572023400.89.91
S4K1284034300.65.74
S4K1338015400.68.87
S4K1372014400.94.97
S4K1444014400.82.89
S4K1579013300.90.94
S5K1296013400.75.84
S5K1303013400.83.87
S5K1310014400.77.76
S5K1439033400.83.87
S5K1580014400.92.94
S6K1345033300.70.81
S6K1350014400.74.88
S6K1398013400.68.75
S6K1438013400.82.79
S6K1582013400.78.90
S6K1582023400.85.94
S6K1582033400.77.89
S6K1582043400.81.88
S7K1327013300.72.82
S7K1354013300.83.90
S7K1361014400.82.89
S7K1361023400.84.88
S7K1361034400.85.92
S7K1583013300.68.76
S8K1272015400.77.90
S8K1364013300.85.92
S8K1371024300.88.97
S8K1376515400.96.99
S8K1424013400.91.95
S8K1584013300.88.90
S8K1584023300.83.88
S9K1733013300.90.92
S9K1766013300.95.96
S9K1776013400.91.97
S9K1776023400.89.94
S10K1725014400.84.91
S10K1729014400.981.00
S10K1747013400.81.89
S10K1797013400.84.85
S11K1258023300.65.71
S11K1258033400.96.98
S11K1277014400.80.90
S11K1417014400.91.96
S11K1442023300.59.73
S12K1287015400.74.88
S12K1290013400.71.70
S12K1300014400.83.84
S12K1300024400.91.92
S12K1334013400.77.81
S13K1260015400.69.90
S13K1385014300.92.99
S13K1403013400.80.88
S13K1403033300.84.91
S13K1587013400.94.94
S13K1587023400.91.94
​

Last updated 05 October 2023 (SK)