NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2012, Grade 12 Economics National Main Assessment, By Item and Blo

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2012, Grade 12 Economics National Assessment, By Item and Block: 2012

NAEP Technical DocumentationScore range and percent agreement for the constructed-response items from the previous year assessed that were rescored in 2012, grade 12 economics national assessment, by item and block: 2012
Block	Item	Range of response codes	Sample size	Percent exact agreement
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percent exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2012 Economics Assessment.
E3	E403901	1-3	600	94
	E404201	1-3	600	89
	E404601	1-3	600	84
	E405101	1-2	600	86
E5	E407501	1-4	600	89
	E408001	1-3	600	71
	E408401	1-3	600	89
E6	E409701	1-3	600	92
	E410201	1-5	600	73
	E410601	1-3	600	92
E7	E411401	1-3	600	79
	E411901	1-4	600	91
	E412301	1-3	600	91
E8	E413201	1-4	600	78
	E413701	1-3	600	95
	E414101	1-2	600	90
E9	E415201	1-3	600	69
	E415701	1-2	600	91
	E416101	1-3	600	90
E10	E417201	1-3	600	86
	E417701	1-5	600	71
	E418101	1-3	600	76

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year assessed that were rescored in 2012, grade 12 economics national assessment, by item and block: 2012
Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
† Not applicable. The intraclass correlation is not reported for dichotomously scored items; Cohen’s Kappa is not reported for polytomously scored items. NOTE: The discrepancy in sample sizes compared to the first table is due to the fact that percent agreement is calculated from the entire sample size, while Cohen's Kappa and intraclass correlation statistics exclude those who omit the item. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2012 Economics Assessment.
E3	E403901	1-3	500	†	0.94
	E404201	1-3	500	†	0.81
	E404601	1-3	600	†	0.77
	E405101	1-2	600	0.62	†
E5	E407501	1-4	500	†	0.86
	E408001	1-3	500	†	0.64
	E408401	1-3	500	†	0.88
E6	E409701	1-3	600	†	0.89
	E410201	1-5	500	†	0.76
	E410601	1-3	600	†	0.92
E7	E411401	1-3	600	†	0.74
	E411901	1-4	500	†	0.87
	E412301	1-3	500	†	0.89
E8	E413201	1-4	500	†	0.82
	E413701	1-3	500	†	0.86
	E414101	1-2	500	0.61	†
E9	E415201	1-3	500	†	0.61
	E415701	1-2	600	0.80	†
	E416101	1-3	500	†	0.90
E10	E417201	1-3	500	†	0.74
	E417701	1-5	500	†	0.82
	E418101	1-3	500	†	0.74

Last updated 30 June 2014 (GF)