NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 8 Science Combined National and State Assessments, by

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 8 Science Combined National and State Assessments, by Item and Block: 2011

NAEP Technical DocumentationScore range, percent agreement for the constructed-response items from the previous year assessed that were rescored in 2009, grade 8 science combined national and state assessments, by item and block: 2011
Block	Item	Range of response codes	Sample size	Percent exact agreement
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2011 Science Assessment.
S3	K113201	1 - 4	1600	89
	K113401	1 - 5	1600	80
	K113701	1 - 5	1600	76
	K113901	1 - 3	1600	85
	K134201	1 - 4	1600	94
S4	K114601	1 - 5	1600	89
	K122402	1 - 4	1600	83
	K122602	1 - 3	1600	86
	K122801	1 - 3	1600	87
	K125201	1 - 5	1600	83
S5	K118501	1 - 4	1600	89
	K120802	1 - 3	1600	85
	K121301	1 - 4	1600	93
	K154601	1 - 3	1600	85
	K154701	1 - 5	1600	85
S6	K122101	1 - 5	1600	77
	K123801	1 - 4	1600	87
	K123802	1 - 3	1600	87
	K125003	1 - 3	1600	85
	K125401	1 - 3	1600	82
	K125402	1 - 3	1600	84
S7	K110401	1 - 4	1600	94
	K110501	1 - 3	1600	90
	K110701	1 - 3	1600	89
	K110702	1 - 3	1600	91
	K110703	1 - 3	1600	87
	K121001	1 - 5	1600	84
	K121002	1 - 3	1600	79
	K121003	1 - 3	1600	86
	K123901	1 - 5	1600	77
S8	K112401	1 - 4	1600	94
	K117202	1 - 2	1600	93
	K155601	1 - 5	1600	82
	K155701	1 - 3	1600	80
	K155702	1 - 4	1600	80
S9	K114801	1 - 4	1600	87
	K114802	1 - 3	1600	92
	K115201	1 - 4	1600	85
	K124102	1 - 4	1600	95
	K156101	1 - 4	1600	91
S11	K111801	1 - 3	1600	90
	K111802	1 - 3	1600	84
	K111803	1 - 3	1600	93
	K117001	1 - 4	1600	96
	K117003	1 - 2	1600	90
	K117004	1 - 3	1600	92
	K119902	1 - 4	1600	87
	K123601	1 - 5	1600	88
S12	K112601	1 - 5	1600	85
	K116901	1 - 3	1600	83
	K119505	1 - 3	1600	90
	K123301	1 - 3	1600	89
	K123401	1 - 4	1400	85
	K157001	1 - 3	1600	71

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year assessed that were rescored in 2009, grade 8 science combined national and state assessments, by item and block: 2011
Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2011 Science Assessment.
S3	K113201	1 - 4	1500	0.83	0.94
	K113401	1 - 5	1400	0.71	0.90
	K113701	1 - 5	1400	0.67	0.90
	K113901	1 - 3	1500	0.75	0.83
	K134201	1 - 4	1400	0.90	0.96
S4	K114601	1 - 5	1500	0.85	0.96
	K122402	1 - 4	1500	0.71	0.89
	K122602	1 - 3	1500	0.71	0.83
	K122801	1 - 3	1500	0.77	0.84
	K125201	1 - 5	1500	0.76	0.93
S5	K118501	1 - 4	1500	0.83	0.93
	K120802	1 - 3	1500	0.70	0.72
	K121301	1 - 4	1500	0.86	0.96
	K154601	1 - 3	1500	0.75	0.84
	K154701	1 - 5	1500	0.73	0.90
S6	K122101	1 - 5	1400	0.68	0.90
	K123801	1 - 4	1400	0.81	0.93
	K123802	1 - 3	1500	0.79	0.87
	K125003	1 - 3	1500	0.76	0.84
	K125401	1 - 3	1500	0.66	0.75
	K125402	1 - 3	1500	0.73	0.83
S7	K110401	1 - 4	1400	0.90	0.97
	K110501	1 - 3	1500	0.83	0.88
	K110701	1 - 3	1500	0.80	0.88
	K110702	1 - 3	1400	0.85	0.91
	K110703	1 - 3	1400	0.74	0.86
	K121001	1 - 5	1400	0.73	0.91
	K121002	1 - 3	1400	0.63	0.75
	K121003	1 - 3	1300	0.72	0.82
	K123901	1 - 5	1400	0.66	0.89
S8	K112401	1 - 4	1500	0.86	0.92
	K117202	1 - 2	1500	0.84	†
	K155601	1 - 5	1500	0.69	0.88
	K155701	1 - 3	1500	0.57	0.72
	K155702	1 - 4	1400	0.60	0.82
S9	K114801	1 - 4	1400	0.76	0.91
	K114802	1 - 3	1400	0.86	0.94
	K115201	1 - 4	1500	0.77	0.92
	K124102	1 - 4	1500	0.92	0.97
	K156101	1 - 4	1500	0.81	0.88
S11	K111801	1 - 3	1500	0.82	0.87
	K111802	1 - 3	1400	0.69	0.84
	K111803	1 - 3	1500	0.84	0.92
	K117001	1 - 4	1400	0.93	0.98
	K117003	1 - 2	1600	0.72	†
	K117004	1 - 3	1400	0.79	0.90
	K119902	1 - 4	1400	0.80	0.84
	K123601	1 - 5	1400	0.83	0.94
S12	K112601	1 - 5	1400	0.73	0.85
	K116901	1 - 3	1400	0.70	0.84
	K119505	1 - 3	1500	0.82	0.88
	K123301	1 - 3	1400	0.80	0.84
	K123401	1 - 4	1300	0.77	0.93
	K157001	1 - 3	1400	0.46	0.56

Last updated 06 January 2016 (GF)

Printer-friendly Version