NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 U.S. History Assessment, by Item and Block : 2006

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 U.S. History Assessment, by Item and Block: 2006

NAEP Technical DocumentationRange of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 U.S. history assessment, by item and block: 2006
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa	Intraclass correlation
H3	H035801	1–3	500	92.00	†	0.80
	H035901	1–3	600	92.00	†	0.90
	H035902	1–3	500	86.00	†	0.82
	H036101	1–3	500	90.00	†	0.91
	H036402	1–4	500	78.00	†	0.82
H4	H059001	1–3	500	88.00	†	0.90
	H059201	1–3	500	76.00	†	0.79
	H059701	1–3	500	92.00	†	0.92
	H059801	1–3	500	87.00	†	0.90
	H060201	1–3	500	83.00	†	0.85
H5	H038103	1–4	400	83.00	†	0.71
	H038301	1–3	500	93.00	†	0.93
	H038601	1–3	500	88.00	†	0.76
	H038702	1–3	500	88.00	†	0.82
	H039001	1–3	500	95.00	†	0.93
H6	H069001	1–3	500	74.00	†	0.76
	H069401	1–3	500	92.00	†	0.92
	H069601	1–3	500	89.00	†	0.84
	H069801	1–4	500	87.00	†	0.90
	H070001	1–2	500	96.00	0.91	†
H7	H057501	1–3	600	97.00	†	0.95
	H057701	1–3	500	87.00	†	0.81
	H057801	1–3	500	83.00	†	0.83
	H058601	1–3	500	85.00	†	0.82
	H058701	1–3	500	93.00	†	0.92
H8	H034101	1–4	600	88.00	†	0.81
	H034401	1–3	500	96.00	†	0.95
	H034501	1–2	600	99.00	0.97	†
	H034702	1–3	500	94.00	†	0.91
	H035001	1–3	500	82.00	†	0.84
	H035101	1–3	500	92.00	†	0.91
H9	H060701	1–3	500	92.00	†	0.91
	H061501	1–3	600	92.00	†	0.91
	H061601	1–3	500	91.00	†	0.86
	H061801	1–3	500	96.00	†	0.93
H10	H042201	1–3	500	95.00	†	0.85
	H042801	1–4	500	89.00	†	0.86
	H042902	1–3	500	92.00	†	0.93
	H043001	1–3	500	89.00	†	0.82
	H043101	1–3	500	94.00	†	0.93
H11	H070401	1–3	500	82.00	†	0.81
	H070701	1–4	500	95.00	†	0.96
	H070801	1–3	500	88.00	†	0.83
	H071001	1–4	600	84.00	†	0.81
	H071601	1–3	500	81.00	†	0.79
H12	H071901	1–3	500	89.00	†	0.91
	H072601	1–3	500	92.00	†	0.82
	H072701	1–4	500	95.00	†	0.97
	H073001	1–3	500	87.00	†	0.90
	H073101	1–3	500	81.00	†	0.83
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2006 U.S. History Assessment.

Last updated 19 November 2009 (GF)

Printer-friendly Version