NAEP Analysis and Scaling - Range of Response Codes, Percentage Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 17 Mathematics Long-Term Trend Assessment: 2004

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 17 Mathematics Long-Term Trend Assessment: 2004

Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 17 mathematics long-term trend assessment: 2004
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa
M21	N307301	1–2	900	99	0.97
	N307701	1–2	900	100	0.99
	N308101	1–2	900	99	0.98
	N308501	1–2	900	98	0.96
M22	N302901	1–2	900	99	0.93
	N308901	1–2	900	99	0.98
	N309301	1–2	900	99	0.97
	N309901	1–2	900	99	0.98
	N310201	1–2	900	100	0.99
M23	N316701	1–2	900	100	0.98
	N318901	1–2	900	99	0.97
	N319301	1–2	900	100	1.00
	N319302	1–2	900	99	0.99
	N319303	1–2	900	100	1.00
	N320101	1–2	900	100	1.00
	N320301	1–2	800	98	0.97
	N320302	1–2	800	99	0.97
	N320401	1–2	800	100	0.99
	N320601	1–2	800	99	0.97
M24	N315501	1–2	800	99	0.98
	N321001	1–2	900	99	0.99
	N321101	1–2	800	100	1.00
	N321401	1–2	700	100	0.99
	N321901	1–2	700	100	0.99
MX51	N303801	1–2	900	100	0.99
	N304101	1–2	900	100	1.00
	N304201	1–2	900	99	0.96
	N304601	1–2	900	99	0.99
	N304901	1–2	900	100	1.00
MX52	N299001	1–2	900	99	0.98
	N305501	1–2	900	99	0.99
	N305801	1–2	900	100	0.99
	N306201	1–2	800	98	0.96
	N306401	1–2	900	100	0.99
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version

Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.