NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 4 Civics National Assessment, by Item and Block: 2010

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2010, Grade 4 Civics National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2010, grade 4 civics national assessment, by item and block: 2010
Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 Civics Assessment.
C4	P040102	1-3	700	84	†	0.84
	P040105	1-3	700	81	†	0.77
	P040109	1-3	700	91	†	0.81
	P040111	1-3	700	73	†	0.73
C5	P040203	1-4	700	86	†	0.82
	P040206	1-3	700	93	†	0.80
	P040209	1-3	700	71	†	0.66
C6	P042122	1-3	700	80	†	0.80
	P042125	1-3	700	83	†	0.71
	P042129	1-3	700	85	†	0.82
	P042132	1-3	700	80	†	0.74
C7	P042136	1-3	700	75	†	0.78
	P042140	1-4	700	83	†	0.91
	P042143	1-4	700	78	†	0.87
	P042147	1-4	700	76	†	0.71

Last updated 06 January 2016 (GF)