NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 Geography National Assessment, by Item and Block: 2010

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 Geography National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 12 geography national assessment, by item and block: 2010
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 Geography Assessment.
G3	G020701	1-3	600	84	†	0.83
	G021001	1-4	600	83	†	0.79
	G021401	1-4	500	87	†	0.95
	G021601	1-4	300	89	†	0.83
	G021602	1-3	500	95	†	0.94
G4	G037501	1-2	600	100	0.99	†
	G037601	1-2	600	100	0.99	†
	G037701	1-2	600	99	0.98	†
	G037801	1-2	600	99	0.98	†
	G037901	1-2	600	98	0.96	†
	G038101	1-3	600	96	†	0.95
	G038401	1-3	600	89	†	0.89
	G038801	1-4	500	88	†	0.89
	G039201	1-3	500	92	†	0.88
G5	G054002	1-3	500	91	†	0.92
	G055201	1-4	600	81	†	0.87
	G055301	1-4	600	99	†	0.99
	G055501	1-3	500	87	†	0.81
G6	G034801	1-4	600	97	†	0.98
	G035001	1-4	500	82	†	0.85
	G035301	1-4	600	89	†	0.93
	G035501	1-4	500	76	†	0.85
G7	G056501	1-3	300	93	†	0.86
	G056801	1-4	600	91	†	0.93
	G057601	1-3	400	92	†	0.90
	G058801	1-3	500	95	†	0.90
	G059001	1-4	500	94	†	0.96
G8	G025001	1-3	500	85	†	0.72
	G025101	1-4	500	67	†	0.77
	G025202	1-3	500	85	†	0.81
	G025301	1-4	400	83	†	0.51
	G025601	1-3	600	95	†	0.91
	G025801	1-3	500	92	†	0.86
G9	G026101	1-3	600	88	†	0.89
	G026204	1-3	600	92	†	0.95
	G026301	1-4	600	89	†	0.86
	G026502	1-3	600	96	†	0.97
	G026503	1-4	600	96	†	0.96
	G026601	1-3	600	99	†	0.98
	G026901	1-3	500	98	†	0.98

Last updated 06 January 2016 (GF)

Printer-friendly Version