NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2007, Grade 4 Reading Combined National and State Assessment, by I

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2007, Grade 4 Reading Combined National and State Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2007, grade 4 reading combined national and state main assessment, by item and block: 2007
Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. ¹ New items introduced in 2006 Precalibration that were rescored in 2007. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Reading Assessment.
R3¹	R053901	1–3	1,500	78.97	—	0.77
	R054001	1–3	1,500	76.39	—	0.66
	R054201	1–4	1,500	67.70	—	0.83
	R054301	1–3	1,500	79.81	—	0.71
R4	R052904	1–3	1,800	83.74	—	0.82
	R052906	1–2	1,500	90.58	0.80	—
	R052907	1–4	1,800	68.41	—	0.77
	R052909	1–2	1,400	91.72	0.78	—
R5	R012601	1–2	1,900	87.15	0.71	—
	R012604	1–2	1,400	92.86	0.81	—
	R012607	1–4	1,400	80.00	—	0.79
	R012612	1–2	1,400	91.19	0.82	—
R6	R017301	1–2	1,600	87.75	0.67	—
	R017303	1–3	1,500	86.66	—	0.88
	R017307	1–4	1,400	74.24	—	0.74
	R017309	1–3	1,400	82.04	—	0.84
	R017310	1–3	1,500	94.74	—	0.95
R7¹	R034601	1–3	1,400	84.47	—	0.87
	R034901	1–3	1,500	81.79	—	0.82
	R035001	1–3	1,400	80.33	—	0.82
	R035201	1–3	1,400	87.17	—	0.85
	R035301	1–4	1,400	81.99	—	0.87
R8¹	R054801	1–3	1,400	92.46	—	0.95
	R055101	1–4	1,400	82.18	—	0.91
	R055301	1–3	1,500	79.78	—	0.79
	R055401	1–3	1,400	88.84	—	0.86
R9	R053003	1–2	1,600	96.69	0.92	—
	R053004	1–3	1,600	89.04	—	0.90
	R053006	1–4	1,500	78.52	—	0.91
	R053009	1–2	1,400	90.00	0.71	—
	R053010	1–3	2,200	85.99	—	0.84
R10	R053101	1–2	1,700	86.98	0.63	—
	R053105	1–3	1,700	80.96	—	0.83
	R053106	1–4	1,500	83.93	—	0.93
	R053108	1–3	1,500	87.23	—	0.84
R11	R020401	1–3	1,600	89.95	—	0.89
	R020701	1–2	1,500	91.84	0.78	—
	R021201	1–3	1,400	80.11	—	0.77
	R021701	1–3	1,500	74.29	—	0.68
R12	R023201	1–2	1,400	87.35	0.74	—
	R023501	1–3	1,500	82.35	—	0.87
	R023601	1–3	1,800	83.57	—	0.83
	R024001	1–3	1,400	78.90	—	0.75

Last updated 02 March 2010 (JL)

Printer-friendly Version