NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2007, Grade 8 Writing Combined National and State Main Assessment

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2007, Grade 8 Writing Combined National and State Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed–response items from previous assessment years, rescored in 2007, grade 8 writing combined national and state main assessment, by item and block: 2007
Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Writing Assessment.
W3	W006002	1–6	1,300	53.45	—	0.78
W5	W006202	1–6	1,400	53.58	—	0.82
W6	W006302	1–6	1,400	52.12	—	0.80
W7	W006402	1–6	1,300	56.54	—	0.85
W8	W006502	1–6	1,400	57.14	—	0.82
W9	W010902	1–6	1,400	55.95	—	0.77
W11	W006802	1–6	1,400	51.15	—	0.79
W12	W006902	1–6	1,400	55.94	—	0.77
W13	W007002	1–6	1,400	52.94	—	0.78
W14	W007102	1–6	1,400	56.08	—	0.78
W15	W011002	1–6	1,400	59.61	—	0.81
W17	W007402	1–6	1,400	56.24	—	0.82
W19	W011102	1–6	1,400	52.08	—	0.76
W21	W007802	1–6	1,400	54.55	—	0.80
W22	W007902	1–6	1,300	52.02	—	0.76
W23	W008002	1–6	1,300	57.97	—	0.78
W24	W008102	1–6	1,400	50.22	—	0.78

Last updated 02 March 2010 (JL)

Printer-friendly Version