NAEP Analysis and Scaling - Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 13 Mathematics Long-Term Trend Bridge Study: 2004

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 13 Mathematics Long-Term Trend Bridge Study: 2004

NAEP Technical DocumentationRange of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 13 mathematics long-term trend bridge study: 2004
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa
M1	N257601	1 - 2	600	99	0.98
	N263101	1 - 2	600	99	0.98
	N275001	1 - 2	600	98	0.95
	N276801	1 - 2	700	100	0.89
	N276802	1 - 2	700	100	0.97
	N276803	1 - 2	600	99	0.97
	N277601	1 - 2	600	100	1.00
	N277602	1 - 2	600	99	0.98
	N277603	1 - 2	600	99	0.96
M2	N256101	1 - 2	700	98	0.87
	N269201	1 - 2	700	97	0.94
	N277901	1 - 2	800	100	0.97
	N277902	1 - 2	800	99	0.95
	N277903	1 - 2	800	99	0.97
	N286601	1 - 2	700	98	0.96
	N286602	1 - 2	700	98	0.96
	N286603	1 - 2	700	99	0.98
M21	N292401	1 - 2	1,400	100	0.99
	N300201	1 - 2	1,300	99	0.98
	N300401	1 - 2	1,300	100	1.00
	N300501	1 - 2	1,400	100	0.99
	N301501	1 - 2	1,400	100	0.99
M22	N301801	1 - 2	800	100	0.96
	N302401	1 - 2	700	99	0.98
	N302601	1 - 2	700	100	1.00
	N302901	1 - 2	700	100	0.98
	N303201	1 - 2	700	100	1.00
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)