Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2008, Rescored in 2016, Grade 8 Music National Assessment, by Block and Item: 2016

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2008, Rescored in 2016, Grade 8 Music National Assessment, by Block and Item: 2016

Score range and percentage of exact agreement for the constructed-response items from previous year 2008, rescored in 2016, grade 8 music national assessment, by block and item: 2016
Block	Item	Score range	Sample size	Percentage of exact agreement
U3	UC00004	1-2	800	99
	UC00006	1-3	800	82
	UC00010	1-4	800	81
	UC00012	1-2	800	95
	UC000A2	1-2	800	92
	UC000A3	1-3	800	88
	UC000B1	1-3	800	88
	UC000B2	1-2	800	93
	UC000B3	1-3	800	82
	UC00B14	1-3	800	79
	UC00B15	1-3	800	87
	UC00C14	1-3	800	82
	UC00C15	1-3	800	80
	UC00D15	1-3	800	82
U4	UD00007	1-3	800	84
	UD000A5	1-2	800	99
	UD000A6	1-3	800	88
	UD000B5	1-2	800	96
	UD000B6	1-3	800	80
	UD000B9	1-3	800	95
	UD000C5	1-2	800	84
	UD000C6	1-2	800	87
	UD000D5	1-2	800	93
	UD000D6	1-3	800	93
	UD000E5	1-3	800	84
	UD000E6	1-3	800	91
U5	UE00004	1-4	800	97
	UE00010	1-3	800	74
	UE00015	1-3	800	83
	UE00016	1-3	800	78
	UE000A6	1-2	800	97
	UE000A8	1-3	800	93
	UE000A9	1-3	800	86
	UE000B6	1-2	800	97
	UE000B8	1-3	800	94
	UE000B9	1-3	800	83
	UE000C8	1-2	800	97
	UE000C9	1-3	800	86
	UE00B11	1-3	800	92
	UE00B12	1-3	800	93
U6	UF00003	1-4	800	73
	UF00005	1-5	800	87
	UF000B1	1-3	800	80
	UF000B2	1-3	800	94
	UF000H1	1-3	800	90
	UF000L1	1-3	800	81
	UF000N1	1-3	800	94
	UF000P1	1-3	800	84
	UF000T1	1-3	800	82
	UF001A6	1-3	800	94
	UF001B6	1-4	800	83
	UF001C6	1-3	800	80
	UF002A6	1-3	800	95
	UF002B6	1-4	800	82
	UF002C6	1-3	800	84
	UF003A6	1-3	800	92
	UF003B6	1-4	800	79
	UF003C6	1-3	800	80
	UF004A6	1-3	800	97
	UF004B6	1-4	800	80
	UF004C6	1-3	800	82
NOTE: Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure. In 2016, grades 4 and 12 were not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2016 Music Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2008, rescored in 2016, grade 8 music national assessment, by block and item: 2016
Block	Item	Score range	Sample size	Cohen’s Kappa	Intraclass correlation
U3	UC00004	1-2	800	.98	†
	UC00006	1-3	700	.66	.69
	UC00010	1-4	700	.69	.78
	UC00012	1-2	700	.89	†
	UC000A2	1-2	800	.81	†
	UC000A3	1-3	700	.75	.84
	UC000B1	1-3	800	.60	.68
	UC000B2	1-2	600	.91	†
	UC000B3	1-3	700	.62	.69
	UC00B14	1-3	700	.50	.54
	UC00B15	1-3	700	.69	.67
	UC00C14	1-3	700	.61	.65
	UC00C15	1-3	700	.58	.60
	UC00D15	1-3	700	.53	.57
U4	UD00007	1-3	700	.64	.75
	UD000A5	1-2	800	.97	†
	UD000A6	1-3	800	.81	.89
	UD000B5	1-2	800	.93	†
	UD000B6	1-3	800	.69	.80
	UD000B9	1-3	800	.91	.93
	UD000C5	1-2	800	.65	†
	UD000C6	1-2	800	.67	†
	UD000D5	1-2	800	.85	†
	UD000D6	1-3	800	.83	.89
	UD000E5	1-3	700	.48	.60
	UD000E6	1-3	700	.78	.80
U5	UE00004	1-4	700	.96	.99
	UE00010	1-3	800	.57	.70
	UE00015	1-3	800	.67	.65
	UE00016	1-3	700	.62	.78
	UE000A6	1-2	700	.94	†
	UE000A8	1-3	800	.86	.90
	UE000A9	1-3	800	.78	.88
	UE000B6	1-2	500	.99	†
	UE000B8	1-3	700	.87	.90
	UE000B9	1-3	700	.74	.83
	UE000C8	1-2	700	.92	†
	UE000C9	1-3	700	.75	.86
	UE00B11	1-3	800	.87	.92
	UE00B12	1-3	800	.85	.92
U6	UF00003	1-4	800	.60	.77
	UF00005	1-5	700	.64	.76
	UF000B1	1-3	700	.61	.77
	UF000B2	1-3	700	.64	.74
	UF000H1	1-3	700	.83	.89
	UF000L1	1-3	700	.69	.78
	UF000N1	1-3	700	.90	.94
	UF000P1	1-3	600	.44	.59
	UF000T1	1-3	700	.67	.76
	UF001A6	1-3	800	.86	.88
	UF001B6	1-4	800	.68	.80
	UF001C6	1-3	700	.60	.71
	UF002A6	1-3	700	.90	.92
	UF002B6	1-4	700	.67	.78
	UF002C6	1-3	700	.62	.68
	UF003A6	1-3	700	.84	.84
	UF003B6	1-4	700	.59	.75
	UF003C6	1-3	600	.48	.58
	UF004A6	1-3	800	.86	.87
	UF004B6	1-4	700	.64	.74
	UF004C6	1-3	700	.65	.76
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2016 Music Assessment.

Last updated 14 February 2022 (SK)

Printer-friendly Version