• Background
• Purpose of This Report
• Comparison of Results When Accommodations Were Not Permitted and When Accommodations Were Permitted
• Relationship Between Exclusion Rates and Results at the State Level
• Conclusion
• Reference

The National Assessment of Educational Progress (NAEP) is the nation’s only ongoing survey of student achievement in core subject areas. Authorized by Congress, administered by the National Center for Education Statistics (NCES) in the U.S. Department of Education, and overseen by the National Assessment Governing Board (NAGB), NAEP regularly reports to the public on the educational progress of a representative sample of students in grades 4, 8, and 12.

Because NAEP’s purpose is to report on what students know and can do, it is important that its student samples and assessment results represent the performance of all students. This includes the results for special-needs students—students with disabilities (SD) and limited-English-proficient (LEP) students. Although the intent of NAEP has consistently been to include special-needs students in its assessments to the fullest degree possible, the implementation of assessments has resulted in some exclusion of SD and LEP students. In order to participate in the NAEP assessments, some special-needs students require accommodations in the test administration. In 1996, NAEP began offering accommodations on a trial basis and conducting research to explore possible psychometric effects that the inclusion of accommodated special-needs students might have on assessment results in various subject areas.

The NAEP 1998 Reading Report Card for the Nation and the States (Donahue et al.1999) included national results for fourth-, eighth-, and twelfth-graders, as well as results for fourth- and eighth-graders in those states and other jurisdictions (i.e., U.S. territories and the District of Columbia) that volunteered to participate in state-level assessments.* In order to allow comparisons with results in 1992 and 1994, when accommodations were not offered, the report card did not incorporate the 1998 results for special-needs students who were tested with accommodations.

There are two purposes to this report. The first is to present NAEP 1998 Reading Assessment results that are recalculated to include results from special-needs students who were tested with accommodations. The second is to examine the impact on NAEP results of the varying exclusion rates of special-needs students, by participating states and other jurisdictions. A follow-up report will explore the patterns of results for accommodated versus nonaccommodated samples separately for students who are SD and LEP, using combined national and state data.

Performance results for both the national and state assessments are reported in two ways. The first is in terms of average scale scores on the NAEP reading composite scale, which ranges from 0 to 500. The second is in terms of percentages of students at or above each of NAGB’s three achievement levels (Basic, Proficient, and Advanced). The following is a summary of the findings addressed in this report:

• There were no statistically significant differences between the originally reported national average scale scores (where accommodations were not permitted) and the recalculated average scores (including data from the administrations where accommodations were permitted) at any of the three grades (table A). Further, there were no statistically significant differences in the percentage of students at or above the Basic or Proficient achievement levels for the two samples at any of the grades assessed.
• In contrast to the unchanged results for the national data, at grade 4, average scale scores were higher in nine states for the original samples without accommodations permitted than for the samples with accommodations permitted (table B). At grade 8, there were no statistically significant differences in average scale scores between the two sample types in any state or jurisdiction. At both grades 4 and 8, there were no statistically significant differences in the percentage of students at or above the Basic or Proficient achievement levels in any state or jurisdiction.
• There were no statistically significant differences in national average reading scale scores between the two sample types at any grade for either male or female students.
• At the state level, however, average reading scores at grade 4 were higher for male students in three states when accommodations were not permitted, higher for female students in four states when accommodations were not permitted, and higher for both male and female students in one state when accommodations were not permitted. No statistically significant differences in the sample types by gender were found in grade 8.
• No statistically significant differences were found in the national data between the two sample types for any ethnic group in any grade.
• The state-level results showed statistically significant differences by race between the two sample types in five states at grade 4. In four states, Black students in the accommodated samples had a lower average scale score than their peers in the samples where accommodations were not permitted. In three states, White students in the accommodated samples had a lower average scale score than their peers in the samples where accommodations were not permitted. In two of the five states noted above, both Black students and White students at grade 4 had lower average scale scores when accommodations were permitted. At grade 8, there were no statistically significant differences between the two sample types by ethnic group in any state or jurisdiction.
• In three states, fourth-graders who were eligible for free/reduced-price lunch under the National School Lunch Program had higher average reading scores when accommodations were not permitted than when accommodations were permitted. At grade 8, there were no statistically significant differences in the average scores of students in the two sample types by eligibility for free/reduced-price school lunch.

Evolving policies and practices regarding the inclusion of special-needs students pose challenges to the state NAEP program as it strives to monitor accurately trends in academic achievement. As policies and practices have changed, the state NAEP program has seen corresponding changes in the proportion of special-needs students included in its samples. This report provides data on statistically significant differences in exclusion rates when accommodations were not permitted and when accommodations were permitted, and the relationship between those exclusion rates and state average scale scores.

• Allowing accommodations in the reading assessment resulted in decreased exclusion rates for a number of participating states and other jurisdictions (table C). At grade 4, 12 of the 43 states or other jurisdictions that participated in the 1998 reading assessment had exclusion percentages of 10 percent or higher when accommodations were not permitted. In contrast, only five jurisdictions had exclusion percentages as high when accommodations were permitted. Differences in exclusion rates by accommodation were statistically significant in 10 of 43 jurisdictions at grade 4 and 10 of 40 jurisdictions at grade 8.
• While no state or other jurisdiction tested large percentages of students with accommodations, considerable variability was evident. At grade 4, two jurisdictions tested more than 5 percent of students with accommodations, while two jurisdictions tested 1 percent or less. At grade 8, there was generally less use of accommodations, although 7 of 40 participating jurisdictions tested 4 percent or more of students with accommodations.
• There is a negative relationship between the change in inclusion rates and difference in average scale scores. States or other jurisdictions that exhibited larger gains in inclusion (i.e., lower exclusion rates) tended to also exhibit larger reduction in their average scale scores when accommodations were permitted. The correlation was -0.72 at fourth grade and
  -0.52 at eighth grade.

Table C. Reading exclusion rates by state for special-needs students, when accommodations were not permitted and when accommodations were permitted, grades 4 and 8: 1998

Offering accommodations in state NAEP to students who receive them in their regular classroom assessments increases inclusion in some states and other jurisdictions, but the magnitude of the increase varies across jurisdictions. At grade 4, the increase in inclusion of special-needs students and the provision of accommodations was associated with lower average scale scores in nine states, but not in the nation. At grade 8, there was no pattern of statistically significant differences by accommodation status.

Readers are cautioned not to overgeneralize the results of this study for several reasons. First, within the reading subject area, the lack of difference between samples with accommodations permitted and those without accommodations permitted in the national data in 1998 did not continue in the 2000 fourth-grade national assessment, where a statistically significant difference did appear in favor of the sample without accommodations. Second, patterns may vary depending on the academic subject area. For example, in the present discussion of 1998 state reading assessment results, more states at grade 4 had lower average scale scores for their accommodated samples than at grade 8. However, for the mathematics assessment in 2000, the findings were different—more states at grade 8 had statistically significant lower average scale scores in accommodated samples than at grade 4. Finally, the fact that the reading assessment did not permit a Spanish translation suggests that the findings for LEP students may not generalize to other subjects, such as mathematics, where this accommodation may be offered.