Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Civics National Assessment, by Item and Block: 2014

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Civics National Assessment, by Item and Block: 2014

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Civics National Assessment, by Item and Block: 2014
Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 8 civics national assessment, by item and block: 2014
Block	Item	Range of response codes	Sample size	Percentage of exact agreement
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percentage of Exact Agreement measure. In 2014, grades 4 and 12 were not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2014 Civics Assessment.
C3	P042171	1-3	600	88
C3	P042174	1-3	600	87
C3	P042177	1-3	600	87
C3	P042181	1-3	600	82
C3	P042184	1-3	600	76
C4	P051401	1-3	600	86
C4	P052101	1-4	600	78
C4	P053801	1-3	600	82
C4	P054101	1-3	600	84
C5	P040903	1-3	600	64
C5	P040906	1-3	600	91
C5	P040910	1-3	600	82
C5	P040913	1-3	600	94
C6	P049201	1-3	600	79
C6	P049501	1-3	600	78
C6	P050501	1-4	600	81
C6	P054801	1-3	600	96
C6	P062201	1-3	600	89
C7	P041102	1-3	600	91
C7	P041106	1-4	600	75
C7	P041111	1-3	600	85
C7	P041116	1-3	600	78
C8	P041202	1-3	600	80
C8	P041205	1-3	600	79
C8	P041213	1-4	600	81
C9	P041307	1-4	600	88
C9	P041309	1-3	600	88
C9	P041315	1-3	600	83
C10	P042152	1-3	600	92
C10	P042155	1-4	600	81
C10	P042158	1-3	600	90
C10	P042162	1-3	600	82

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 8 civics national assessment, by item and block: 2014
Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. In 2014, grades 4 and 12 were not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2014 Civics Assessment.
C3	P042171	1-3	600	.81	.89
C3	P042174	1-3	500	.55	.59
C3	P042177	1-3	600	.79	.89
C3	P042181	1-3	500	.69	.79
C3	P042184	1-3	500	.57	.68
C4	P051401	1-3	500	.75	.83
C4	P052101	1-4	500	.62	.78
C4	P053801	1-3	600	.59	.61
C4	P054101	1-3	500	.63	.71
C5	P040903	1-3	600	.40	.56
C5	P040906	1-3	400	.73	.85
C5	P040910	1-3	500	.64	.71
C5	P040913	1-3	500	.89	.93
C6	P049201	1-3	600	.65	.78
C6	P049501	1-3	500	.59	.72
C6	P050501	1-4	500	.72	.86
C6	P054801	1-3	400	.91	.94
C6	P062201	1-3	400	.77	.83
C7	P041102	1-3	500	.81	.87
C7	P041106	1-4	500	.59	.70
C7	P041111	1-3	500	.68	.79
C7	P041116	1-3	500	.53	.62
C8	P041202	1-3	600	.68	.79
C8	P041205	1-3	600	.50	.52
C8	P041213	1-4	500	.68	.84
C9	P041307	1-4	500	.79	.90
C9	P041309	1-3	500	.78	.87
C9	P041315	1-3	500	.71	.82
C10	P042152	1-3	600	.87	.92
C10	P042155	1-4	500	.71	.87
C10	P042158	1-3	500	.82	.86
C10	P042162	1-3	500	.65	.77

Last updated 07 May 2021 (SK)

Printer-friendly Version

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Civics National Assessment, by Item and Block: 2014