NAEP Analysis and Scaling - Range of Response Codes, Percentage Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 9 Mathematics Long-Term Trend Assessment: 2004

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 9 Mathematics Long-Term Trend Assessment: 2004

Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 9 mathematics long-term trend assessment: 2004
Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa
M21	N292101	1–2	900	100	1.00
	N292401	1–2	900	100	1.00
	N293001	1–2	900	100	1.00
	N293301	1–2	900	100	0.99
	N293501	1–2	900	100	1.00
M22	N294201	1–2	900	100	0.98
	N294301	1–2	900	99	1.00
	N295301	1–2	900	100	0.99
	N295401	1–2	900	99	0.98
	N295701	1–2	800	100	0.99
M23	N310401	1–2	900	100	0.95
	N310701	1–2	900	100	0.96
	N310702	1–2	900	100	0.98
	N310703	1–2	900	99	0.98
	N311101	1–2	900	100	1.00
	N311201	1–2	900	100	1.00
	N311301	1–2	900	100	0.99
	N311501	1–2	900	100	0.99
	N311502	1–2	800	99	0.98
M24	N312901	1–2	900	99	0.98
	N313001	1–2	900	100	0.99
	N313101	1–2	900	99	0.98
	N313102	1–2	900	99	0.98
	N313103	1–2	900	99	0.99
	N313701	1–2	800	100	0.99
	N313702	1–2	700	99	0.99
	N313801	1–2	800	100	0.99
	N313901	1–2	800	100	0.99
MX51	N287601	1–2	800	99	0.98
	N288101	1–2	900	100	0.99
	N288501	1–2	800	100	0.99
	N289201	1–2	800	100	1.00
MX52	N289701	1–2	900	100	1.00
	N289901	1–2	900	100	0.98
	N290101	1–2	900	100	0.99
	N290701	1–2	900	100	0.98
	N291501	1–2	800	100	1.00
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Last updated 06 April 2009 (GF)

Printer-friendly Version

Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.