NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Civics National Assessment, by Item and Block: 2010

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Civics National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 civics national assessment, by item and block: 2010
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 Civics Assessment.
C3	P042901	1-3	600	80	†	0.80
	P043301	1-3	600	86	†	0.81
	P047401	1-3	600	88	†	0.85
	P048001	1-4	500	88	†	0.91
C4	P040102	1-3	600	91	†	0.92
	P040105	1-3	600	85	†	0.84
	P040109	1-3	500	95	†	0.93
	P040111	1-3	500	86	†	0.88
C5	P040203	1-4	600	92	†	0.92
	P040206	1-3	600	95	†	0.82
	P040209	1-3	500	88	†	0.85
C6	P042122	1-3	600	83	†	0.84
	P042125	1-3	600	88	†	0.80
	P042129	1-3	600	88	†	0.85
	P042132	1-3	500	85	†	0.81
C7	P042136	1-3	600	80	†	0.82
	P042140	1-4	600	84	†	0.91
	P042143	1-4	600	87	†	0.93
	P042147	1-4	500	87	†	0.86
C8	P044301	1-3	500	90	†	0.82
	P044801	1-4	500	80	†	0.80
	P045501	1-3	600	84	†	0.81

Last updated 06 January 2016 (GF)