Skip to main content

Increasing the Participation of Special Needs Students in NAEP:

A Report on 1996 NAEP Research Activities

February 2000

Authors: John Mazzeo, James E. Carlson, Kristin E. Voelkl, and Anthony D. Lutkus

PDF Download the complete report in a PDF file for viewing and printing. 1,936K

Executive Summary

This study grew out of concerns about the underrepresentation of students with special needs in the National Assessment of Educational Progress (NAEP) assessments. The term "special needs students" is sometimes used to include both students with disabilities and students who are limited English proficient (LEP). In the 1996 NAEP assessment samples, 10 percent of fourth graders, 9 percent of eighth graders, and 5 percent of twelfth graders were identified by their schools as students with disabilities. In the same assessment year, 4 percent of fourth graders and 2 percent of eighth and twelfth graders were identified by their schools as students with limited English proficiency.[1] Schools participating in NAEP have been permitted to exclude individuals they identify as special needs students from the assessment, in accordance with criteria provided by the program at that time. In fact, at least half of all special needs students were excluded from NAEP assessments in 1992 and 1994. This exclusion has raised concerns that some special needs students who could be meaningfully assessed are being excluded from NAEP. Moreover, there is an additional concern that variations across locales in exclusion practices may introduce biases in NAEP results.

In recent years, a number of policy, legislative, civil rights, and technical considerations have caused the NAEP program to look more closely at its administration and assessment procedures and to consider changes that can increase participation among students with disabilities and LEP students. Based on previous studies,[2] as well as recommendations from various offices in the U.S. Department of Education, program procedures have been modified with the aim of increasing participation among special needs students. Modifications were made in two areas.[3] First, inclusion criteria for the NAEP 1996 assessment were revised with the intention of making them clearer, more inclusive, and more likely to be applied consistently across jurisdictions participating in the state assessment program. Second, for the first time in NAEP, a variety of assessment accommodations were offered to 1) students with disabilities whose Individualized Education Plan (IEP) specified such accommodations for testing; and 2) LEP students who, in the opinion of their instructors, required an accommodation in order to take the assessment in English.

This report presents in-depth analyses of the effects on inclusion rates of the above efforts to increase the participation of special needs students in NAEP. It also contains an analysis of selected technical characteristics of the assessment results and a review of descriptive results of the background characteristics and educational experiences of students with disabilities and LEP students who participated in the NAEP 1996 national assessments in mathematics and science.

In particular, data are presented on:

  • the possible effect on the NAEP proficiency scales of including greater percentages of special needs students;

  • the comparability of results from nonstandard administrations (i.e., administrations in which accommodations were allowed) to results obtained under standard conditions; and

  • the effect of nonstandard administrations on NAEP’s capacity to provide accurate comparisons of trends over time.

In addition, it is important to be clear on what this report does not contain:

  • This report does not provide an in-depth examination of the performance on NAEP of students with disabilities and LEP students.

  • The relatively small sample sizes obtained in the study did not allow disaggregation of students with disabilities and LEP students in many of the statistical analyses that dealt with the effects on NAEP scales.

  • This report does not separate students with disabilities from LEP students in the Differential Item Functioning (DIF) analyses.

  • This report does not look at performance results or inclusion rates for students with disabilities and LEP students by state.

A experiment was designed for the 1996 assessments in mathematics and science, which permitted analysis of data relevant to the issues above. In addition, a questionnaire was included that was designed to obtain information on student background and educational experiences. The questionnaire was to be completed for all sampled students with disabilities and for all sampled LEP students. The design of the NAEP 1996 assessment included three distinct national samples of schools. In the first of these school samples (denoted S1), the assessment was conducted using the same inclusion criteria used during the 1990 and 1992 NAEP assessments in mathematics and science. In the second school sample (denoted S2), revised inclusion criteria were used. No assessment accommodations or adaptations were offered to students in S1 or S2 schools. In the third sample (denoted S3), the assessment was conducted using inclusion criteria that were identical to those used in S2 schools. The S3 sample was distinguished, however, by the availability of a variety of assessment accommodations and adaptations. To ensure sufficient amounts of data for planned analyses, students with disabilities and LEP students were oversampled in national S2 and S3 schools, and all students in S3 who received an accommodation at a given grade were administered the same NAEP assessment booklet.


Major Findings

Technical Characteristics of Results

The findings of the current research on technical characteristics of the assessment results based on the combined data from all special-needs students include the following:

  • For two of the three grades in science there is some evidence to suggest that test results obtained using accommodations and adaptations cannot be fit with the same Item Response Theory (IRT)[4] model as results obtained under standard administration conditions. The evidence for the mathematics assessment was less conclusive. Because small sample sizes necessitated the combination of students with disabilities and limited English proficient students for IRT and scaling analyses, it is not yet clear whether future NAEP reports will need to report these categories of students separately. A future report using larger samples (combined state data) from the 1998 Reading assessment should shed further light on this question.

  • Despite the finding above, the inclusion of data from nonstandard administrations had no discernable effect on aggregate NAEP scaling results in mathematics and science at any of the three grades. Differences in test characteristic curves and test information curves plotted with and without the inclusion of such data differed no more than would be expected due to sampling variability.

  • Proficiency means were estimated for the NAEP mathematics and science scales, with and without the inclusion of students with accommodations in testing at each of grades 4, 8, and 12. There were no significant differences in the overall means or in the means for significant subgroups at any of the three grades.

  • The results reported here suggest that the procedural changes being considered would not significantly affect the NAEP scale score results. If so, it may be possible for the NAEP program to achieve its joint goals of increasing inclusion while maintaining trend lines. However, additional research is necessary to determine the generality of these findings across content areas and over time, as state policies and procedures with respect to inclusion evolve.

Data from background questionnaires did allow separate analyses for SD and LEP students pertaining to background characteristics, educational experiences, and inclusion rates. Major findings for these analyses are summarized below.


Students with Disabilities

Background Characteristics, Educational Experiences and Inclusion Rates

  • Learning disability was by far the most frequently reported category for students with disabilities, with close to three of four students so identified at each of the three grades.

  • About half of the students at each grade was described as having mild disabilities. The remaining half at each grade was almost all categorized with moderate to severe disabilities. Very few students receiving special educational services at schools participating in NAEP (1 percent at grades four and eight, and 3 percent at grade twelve) were judged to have profound disabilities.

  • Regardless of grade level, about half of all students with disabilities were mainstreamed in academic subjects at least 80 percent of the time.

  • In reading/language arts, half or fewer of the students with disabilities received instruction that was at grade level. In mathematics and science, the situation was slightly better at the two lower grades. More than half of the grade four and grade eight students with disabilities received grade-level instruction, and over 70 percent of these students received grade-level instruction in science.

  • Almost all students who received instruction that was at or above grade level received the same curriculum content as their nondisabled peers. In contrast, fewer than half of those students with disabilities who received below grade-level instruction was taught the same curriculum content as their nondisabled peers.

  • In all three grades, more than 75 percent of students with disabilities were judged by school personnel to be performing below grade level in reading/language arts. Reported performance levels in mathematics and science were somewhat higher than those in reading/language arts at grade four.

  • Across the three grades, respondents reported that 42 to 44 percent of students with disabilities received some form of accommodation or adaptation in testing.

Inclusion Rates

  • Comparison of questionnaire results with actual participation rates from the 1996 mathematics assessment suggest that: 1) increases in the percentages of students with disabilities participating in NAEP are not likely to result solely from revisions to inclusion criteria; and, 2) a further expansion of accommodations or adaptations permitted by NAEP, or a change in NAEP guidelines as to who is eligible for special testing conditions, could result in further small increases in inclusion percentages.

  • Most exclusion decisions were made on the basis of what is stated in the IEP, and relatively few exclusion decisions were made on the basis of a judgment of severe cognitive impairment, absent corroborating direction from the IEP. However, results also suggest that, for substantial percentages of excluded students, neither determination by the IEP team nor the presence of cognitive impairments was given as reason for exclusion.

  • Some students who do not regularly receive accommodations or adaptations were offered them in NAEP and others who should not have been tested were, in fact, included. These results suggest that incorrect decisions regarding inclusion or testing condition may have been made or that incorrect questionnaire data may have been provided.


Students with Limited English Proficiency

Background Characteristics and Educational Experiences

  • The largest proportion of LEP students spoke Spanish as their native language (74 percent at grade four, 72 percent at grade eight, and 54 percent at grade twelve). The most frequently encountered other languages were Vietnamese, Hmong, Chinese, Russian, and Pacific Island languages.

  • Forty-four percent of grade-four LEP students, 47 percent of grade-eight LEP students, and 65 percent of grade-twelve LEP students had received academic instruction primarily in English for three or more years.

  • At grades eight and twelve, few students received native-language instruction in academic areas. At grade four, the percentages of LEP students who received native- language instruction in reading/language arts, mathematics, and science were 22, 27, and 26 percent, respectively.

  • Among LEP students receiving English-language instruction, the majority received instruction at grade level at all three grades.

  • The vast majority of LEP students at all three grades (87 percent of grade-four LEP students, 80 percent of grade-eight LEP students and 81 percent of grade-twelve LEP students) received some special academic instruction in English or in their native language. At grades eight and twelve, such special instruction appears to have been predominantly in English.

  • Although most LEP students were receiving English-language instruction on grade-level, a significant percentage were judged to be performing below grade level in English. In reading/language arts, where one might expect the impact of limited language proficiency to be most pronounced, 70 percent of grade-four and 62 percent of grade-eight LEP students were judged by school personnel as performing below grade level in English; at grade twelve, 50 percent were so judged. In science, the percentages reported performing below grade level ranged from 30 at grade twelve to 44 at grade eight. In mathematics, the percentages ranged from 33 percent (at grade twelve) to 46 percent (at grade eight).

  • Respondents indicated that 37 percent of grade-four LEP students, 27 percent of grade-eight LEP students, and 22 percent of grade-twelve LEP students used accommodations and adaptations in achievement testing in their schools.

Inclusion Rates

  • The operational criteria used in NAEP from 1990 to 1996 indicated that LEP students enrolled in schools where English is the primary language of instruction for two or more years were to be included in the assessment. At least 85 percent of fourth-grade students, 67 percent of eighth-grade students, and 83 percent of twelfth-grade students had been enrolled for two or more years in schools where English was the primary language. Historically, NAEP inclusion rates for LEP students have been below the ideal minimums suggested by questionnaire results.

  • As was the case for students with disabilities, comparisons of questionnaire results with assessment inclusion rates for LEP students suggest that: 1) increases in the percentage of LEP students are not likely to result solely from revisions to inclusion criteria that do not also involve the provision of accommodations; and 2) further modest improvements in inclusion might still be possible if the list of permitted accommodations and adaptations can be expanded.

  • Analyses of inclusion rates by the length of time students were enrolled in schools where English is the primary language of instruction provided some evidence that, when implemented without the provision of accommodations and adaptations, the revised criteria actually resulted in less inclusion among LEP students than did the original criteria. This evidence was strongest at grade four.

  • Under the revised criteria, all students receiving academic instruction in English for three or more years were to be included in NAEP. Analyses based on questionnaire responses as to the number of years students were receiving academic instruction in English indicated that this ideal was not quite achieved. Inclusion rates among students with three or more years of academic instruction in English were high, but total inclusion was not achieved, even where accommodations and adaptations were provided.

  • Some LEP students who do not usually receive accommodations in testing were apparently provided accommodations in the NAEP assessment. The percentages of LEP students in this category were small (10, 6, and 5 percent in grades four, eight, and twelve, respectively).

  • Questionnaire results suggest that the procedural modifications made to NAEP had their primary impact on inclusion rates at grades four and eight among students who would be tested in their native language if this accommodation were available. Participation rates for these students were higher when accommodations were available.
  1. Reese, C.M., Miller, K.E., Mazzeo, J., & Dossey, J.A. (1997). NAEP 1996 mathematics report card for the nation and the states. (p. 67). Washington, DC: National Center for Education Statistics.

  2. National Academy of Education. (1993). The trial state assessment: Prospects and realities; the third report of the national academy of education panel on the evaluation of the NAEP 1992 trial state assessment, Stanford, CA: Author.

  3. Olson, J.F., & Goldstein, A.A. (1996). Increasing the inclusion of students with disabilities and limited English proficient students in NAEP. Focus on NAEP. 2(1). Washington, DC: National Center for Education Statistics.

  4. IRT analyses provides a common scale on which performance can be compared across groups such as those defined by grade and characteristics, including gender and race/ethnicity.

PDF Download the complete report in a PDF file for viewing and printing. 1,936K

NCES 2000-473 Ordering information

Suggested Citation
U.S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics. Increasing the Participation of Special Needs Students in NAEP: A Report on 1996 NAEP Research Activities, NCES 2000-473, by J. Mazzeo, J. E. Carlson, K. E. Voelkl, & A. D. Lutkus. Washington, DC: 2000.

Last updated 14 March 2001 (RH)

Go to Top of Page