NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Main Assessment, by Item and Block

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Main Assessment, by Item and Block: 2007

Range of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 mathematics combined national and state main assessment, by item and block: 2007
Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Mathematics Assessment.
M3	M145901	1–3	1,600	95.41	—	0.97
	M146101	1–2	1,600	96.91	0.93	—
	M146301	1–3	1,500	97.41	—	0.98
	M146401	1–3	1,500	96.36	—	0.96
	M146601	1–5	1,500	82.38	—	0.94
M4	M146801	1–2	1,600	99.10	0.98	—
	M146901	1–3	1,600	98.92	—	0.99
	M147001	1–3	1,500	93.62	—	0.95
	M148101	1–3	1,500	84.62	—	0.78
M5	M019701	1–2	1,500	99.22	0.98	—
	M020001	1–2	1,500	99.20	0.98	—
	M066301	1–3	1,500	96.17	—	0.95
	M066501	1–3	1,500	96.26	—	0.98
	M067901	1–3	1,500	97.54	—	0.98
	M085401	1–5	1,600	86.34	—	0.95
	M085701	1–3	1,600	94.94	—	0.95
	M085901	1–3	1,600	93.21	—	0.93
M6	M136501	1–2	1,600	99.44	0.99	—
	M136601	1–3	1,600	95.30	—	0.97
	M136901	1–4	1,600	95.23	—	0.96
M7	M139301	1–2	1,600	99.02	0.98	—
	M139601	1–3	1,600	98.83	—	0.99
	M139801	1–4	1,500	93.94	—	0.95
	M140001	1–3	1500	96.60	—	0.96
	M140301	1–5	1,500	91.69	—	0.96
M8	M010631	1–3	1,600	98.47	—	0.99
	M040001	1–3	1,500	97.64	—	0.98
	M072401	1–3	1,500	94.83	—	0.96
	M087001	1–3	1,600	96.14	—	0.97
	M087301	1–5	1,400	92.15	—	0.97
	M091201	1–3	1,500	93.59	—	0.95
M9	M040201	1–2	1,500	93.98	0.82	—
	M043201	1–2	1,600	98.02	0.94	—
	M043301	1–3	1,500	98.89	—	0.99
	M043401	1–4	1,600	97.52	—	0.99
	M043402	1–4	1,500	97.64	—	0.99
	M043403	1–3	1,500	97.69	—	0.97
	M066701	1–3	1,600	95.04	—	0.96
	M066801	1–3	1,500	96.73	—	0.96
	M086601	1–3	1,600	98.16	—	0.98
	M091401	1–5	1,500	86.92	—	0.95
M10	M137501	1–2	1,600	98.90	0.97	—
	M138201	1–2	1,600	95.76	0.91	—
	M138401	1–4	1,800	82.80	—	0.83
	M138701	1–5	1,500	90.45	—	0.97
M11	M096001	1–2	1,600	96.03	0.92	—
	M097201	1–3	1,600	93.36	—	0.96
	M097401	1–3	1,600	97.88	—	0.97
M12	M148501	1–3	1,600	98.75	—	0.99
	M149201	1–3	1,500	94.49	—	0.95
	M149301	1–2	1,500	98.66	0.97	—
	M149701	1–5	1,500	91.99	—	0.97

Last updated 02 March 2010 (JL)

Printer-friendly Version

Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.

National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education

U.S. Department of Education
Institute of Education Sciences
National Center for Education Statistics