NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 Economics Assessment, by Item and Block: 2006

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 Economics Assessment, by Item and Block: 2006

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 12 economics assessment, by item and block: 2006
Block	Item	Range of response codes	Sample Size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. Data was not collected or not reported. † The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen’s kappa is a measure of reliability that is used for items that are dichotomously scored. The intraclass correlation is used for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2006 Economics Assessment.
E1	E000301	1–3	500	81.82	†	0.85
	E000701	1–3	500	84.84	†	0.81
	E001101	1–4	500	72.34	†	0.81
	E001501	1–3	400	79.67	†	0.64
E2	E002001	1–3	500	92.88	†	0.86
	E002501	1–4	500	82.18	†	0.91
	E002901	1–3	500	71.51	†	0.64
E3	E003901	1–3	500	91.77	†	0.93
	E004201	1–3	500	87.60	†	0.81
	E004601	1–3	500	86.78	†	0.81
	E005101	1–2	500	85.34	0.62	—
E4	E005701	1–3	500	93.91	†	0.88
	E006201	1–4	500	76.73	†	0.82
	E006601	1–3	500	87.42	†	0.87
E5	E007501	1–4	500	90.70	†	0.89
	E008001	1–3	500	71.12	†	0.67
	E008401	1–3	500	89.61	†	0.87
E6	E009701	1–3	500	91.11	†	0.88
	E010201	1–5	400	72.17	†	0.75
	E010601	1–3	500	94.72	†	0.95
E7	E011401	1–3	600	77.05	†	0.72
	E011901	1–4	500	85.77	†	0.82
	E012301	1–3	400	88.43	†	0.86
E8	E013201	1–4	500	76.83	†	0.82
	E013701	1–3	400	95.50	†	0.89
	E014101	1–2	600	92.32	0.74	—
E9	E015201	1–3	500	72.52	†	0.65
	E015701	1–2	500	90.62	0.80	—
	E016101	1–3	500	89.85	†	0.89
E10	E017201	1–3	500	90.59	†	0.82
	E017701	1–5	500	65.95	†	0.86
	E018101	1–3	400	75.78	†	0.77

Last updated 08 December 2009 (GF)

Printer-friendly Version