NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years Rescored in 2006, Grade 4 Civics Assessment, By Item and Block: 2006

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years Rescored in 2006, Grade 4 Civics Assessment, by Item and Block: 2006

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years rescored in 2006, grade 4 civics assessment, by item and block: 2006
Block	Item	Range of response codes	Sample size	Percent of exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2006 Civics Assessment.
P3	P030004	1–3	800	70	†	0.68
	P030005	1–3	800	83	†	0.80
	P030007	1–3	800	81	†	0.78
	P030010	1–4	700	82	†	0.83
P4	P040102	1–3	800	86	†	0.85
	P040105	1–3	800	82	†	0.80
	P040109	1–3	700	92	†	0.88
	P040111	1–3	700	81	†	0.80
P5	P040203	1–4	800	87	†	0.88
	P040206	1–3	800	90	†	0.69
	P040209	1–3	800	74	†	0.64
P8	P040502	1–4	800	77	†	0.87
	P040507	1–3	800	87	†	0.76
	P040510	1–3	700	75	†	0.69

Last updated 19 November 2009 (GF)