Increasing the Participation of Special Needs Students in NAEP:
A Report on 1996 NAEP Research Activities
Authors: John Mazzeo, James E. Carlson, Kristin E. Voelkl, and Anthony D. Lutkus
Download the complete report in a PDF file for viewing and printing.
This study grew out of concerns about the underrepresentation of students with special needs in
the National Assessment of Educational Progress (NAEP) assessments. The term "special
needs students" is sometimes used to include both students with disabilities and students who
are limited English proficient (LEP). In the 1996 NAEP assessment samples, 10 percent of
fourth graders, 9 percent of eighth graders, and 5 percent of twelfth graders were identified by
their schools as students with disabilities. In the same assessment year, 4 percent of fourth
graders and 2 percent of eighth and twelfth graders were identified by their schools as students
with limited English proficiency. Schools participating in NAEP have been permitted to
exclude individuals they identify as special needs students from the assessment, in accordance
with criteria provided by the program at that time. In fact, at least half of all special needs
students were excluded from NAEP assessments in 1992 and 1994. This exclusion has raised
concerns that some special needs students who could be meaningfully assessed are being
excluded from NAEP. Moreover, there is an additional concern that variations across locales in
exclusion practices may introduce biases in NAEP results.
In recent years, a number of policy, legislative, civil rights, and technical considerations
have caused the NAEP program to look more closely at its administration and assessment
procedures and to consider changes that can increase participation among students with
disabilities and LEP students. Based on previous studies, as well as recommendations from
various offices in the U.S. Department of Education, program procedures have been modified
with the aim of increasing participation among special needs students. Modifications were made
in two areas. First, inclusion criteria for the NAEP 1996 assessment were revised with the
intention of making them clearer, more inclusive, and more likely to be applied consistently
across jurisdictions participating in the state assessment program. Second, for the first time in
NAEP, a variety of assessment accommodations were offered to 1) students with disabilities
whose Individualized Education Plan (IEP) specified such accommodations for testing; and 2)
LEP students who, in the opinion of their instructors, required an accommodation in order to
take the assessment in English.
This report presents in-depth analyses of the effects on inclusion rates of the above
efforts to increase the participation of special needs students in NAEP. It also contains an
analysis of selected technical characteristics of the assessment results and a review of
descriptive results of the background characteristics and educational experiences of students
with disabilities and LEP students who participated in the NAEP 1996 national assessments in
mathematics and science.
In particular, data are presented on:
the possible effect on the NAEP proficiency scales of including greater percentages
of special needs students;
the comparability of results from nonstandard administrations (i.e., administrations in
which accommodations were allowed) to results obtained under standard conditions; and
the effect of nonstandard administrations on NAEPs capacity to provide accurate
comparisons of trends over time.
In addition, it is important to be clear on what this report does not contain:
This report does not provide an in-depth examination of the performance on NAEP of
students with disabilities and LEP students.
The relatively small sample sizes obtained in the study did not allow disaggregation
of students with disabilities and LEP students in many of the statistical analyses that
dealt with the effects on NAEP scales.
This report does not separate students with disabilities from LEP students in the
Differential Item Functioning (DIF) analyses.
This report does not look at performance results or inclusion rates for students with
disabilities and LEP students by state.
A experiment was designed for the 1996 assessments in mathematics and science,
which permitted analysis of data relevant to the issues above. In addition, a questionnaire was
included that was designed to obtain information on student background and educational
experiences. The questionnaire was to be completed for all sampled students with disabilities
and for all sampled LEP students. The design of the NAEP 1996 assessment included three
distinct national samples of schools. In the first of these school samples (denoted S1), the
assessment was conducted using the same inclusion criteria used during the 1990 and 1992
NAEP assessments in mathematics and science. In the second school sample (denoted S2),
revised inclusion criteria were used. No assessment accommodations or adaptations were
offered to students in S1 or S2 schools. In the third sample (denoted S3), the assessment was
conducted using inclusion criteria that were identical to those used in S2 schools. The S3 sample
was distinguished, however, by the availability of a variety of assessment accommodations and
adaptations. To ensure sufficient amounts of data for planned analyses, students with disabilities and
LEP students were oversampled in national S2 and S3 schools, and all students in S3 who received
an accommodation at a given grade were administered the same NAEP assessment booklet.
Technical Characteristics of Results
The findings of the current research on technical characteristics of the assessment
results based on the combined data from all special-needs students include the following:
For two of the three grades in science there is some evidence to suggest that test
results obtained using accommodations and adaptations cannot be fit with the same
Item Response Theory (IRT) model as results obtained under standard
administration conditions. The evidence for the mathematics assessment was less
conclusive. Because small sample sizes necessitated the combination of students
with disabilities and limited English proficient students for IRT and scaling analyses,
it is not yet clear whether future NAEP reports will need to report these categories of
students separately. A future report using larger samples (combined state data) from
the 1998 Reading assessment should shed further light on this question.
Despite the finding above, the inclusion of data from nonstandard administrations had no
discernable effect on aggregate NAEP scaling results in mathematics and science at any
of the three grades. Differences in test characteristic curves and test information curves
plotted with and without the inclusion of such data differed no more than would be
expected due to sampling variability.
Proficiency means were estimated for the NAEP mathematics and science scales, with
and without the inclusion of students with accommodations in testing at each of grades 4,
8, and 12. There were no significant differences in the overall means or in the means for
significant subgroups at any of the three grades.
The results reported here suggest that the procedural changes being considered would not
significantly affect the NAEP scale score results. If so, it may be possible for the NAEP
program to achieve its joint goals of increasing inclusion while maintaining trend lines.
However, additional research is necessary to determine the generality of these findings
across content areas and over time, as state policies and procedures with respect to
Data from background questionnaires did allow separate analyses for SD and LEP
students pertaining to background characteristics, educational experiences, and inclusion rates.
Major findings for these analyses are summarized below.
Students with Disabilities
Background Characteristics, Educational Experiences and Inclusion Rates
Learning disability was by far the most frequently reported category for students with
disabilities, with close to three of four students so identified at each of the three
About half of the students at each grade was described as having mild disabilities.
The remaining half at each grade was almost all categorized with moderate to severe
disabilities. Very few students receiving special educational services at schools
participating in NAEP (1 percent at grades four and eight, and 3 percent at grade
twelve) were judged to have profound disabilities.
Regardless of grade level, about half of all students with disabilities were
mainstreamed in academic subjects at least 80 percent of the time.
In reading/language arts, half or fewer of the students with disabilities received
instruction that was at grade level. In mathematics and science, the situation was
slightly better at the two lower grades. More than half of the grade four and grade
eight students with disabilities received grade-level instruction, and over 70 percent
of these students received grade-level instruction in science.
Almost all students who received instruction that was at or above grade level received
the same curriculum content as their nondisabled peers. In contrast, fewer than half
of those students with disabilities who received below grade-level instruction was
taught the same curriculum content as their nondisabled peers.
In all three grades, more than 75 percent of students with disabilities were judged by
school personnel to be performing below grade level in reading/language arts.
Reported performance levels in mathematics and science were somewhat higher than
those in reading/language arts at grade four.
Across the three grades, respondents reported that 42 to 44 percent of students with
disabilities received some form of accommodation or adaptation in testing.
Comparison of questionnaire results with actual participation rates from the 1996
mathematics assessment suggest that: 1) increases in the percentages of students with
disabilities participating in NAEP are not likely to result solely from revisions to
inclusion criteria; and, 2) a further expansion of accommodations or adaptations
permitted by NAEP, or a change in NAEP guidelines as to who is eligible for special
testing conditions, could result in further small increases in inclusion percentages.
Most exclusion decisions were made on the basis of what is stated in the IEP, and
relatively few exclusion decisions were made on the basis of a judgment of severe
cognitive impairment, absent corroborating direction from the IEP. However, results
also suggest that, for substantial percentages of excluded students, neither
determination by the IEP team nor the presence of cognitive impairments was given
as reason for exclusion.
Some students who do not regularly receive accommodations or adaptations were
offered them in NAEP and others who should not have been tested were, in fact,
included. These results suggest that incorrect decisions regarding inclusion or testing
condition may have been made or that incorrect questionnaire data may have been
Students with Limited English Proficiency
Background Characteristics and Educational Experiences
The largest proportion of LEP students spoke Spanish as their native language
(74 percent at grade four, 72 percent at grade eight, and 54 percent at grade twelve).
The most frequently encountered other languages were Vietnamese, Hmong, Chinese,
Russian, and Pacific Island languages.
Forty-four percent of grade-four LEP students, 47 percent of grade-eight LEP
students, and 65 percent of grade-twelve LEP students had received academic
instruction primarily in English for three or more years.
At grades eight and twelve, few students received native-language instruction in
academic areas. At grade four, the percentages of LEP students who received
native- language instruction in reading/language arts, mathematics, and science were
22, 27, and 26 percent, respectively.
Among LEP students receiving English-language instruction, the majority received
instruction at grade level at all three grades.
The vast majority of LEP students at all three grades (87 percent of grade-four LEP
students, 80 percent of grade-eight LEP students and 81 percent of grade-twelve LEP
students) received some special academic instruction in English or in their native
language. At grades eight and twelve, such special instruction appears to have been
predominantly in English.
Although most LEP students were receiving English-language instruction on
grade-level, a significant percentage were judged to be performing below grade level
in English. In reading/language arts, where one might expect the impact of limited
language proficiency to be most pronounced, 70 percent of grade-four and 62 percent
of grade-eight LEP students were judged by school personnel as performing below
grade level in English; at grade twelve, 50 percent were so judged. In science, the
percentages reported performing below grade level ranged from 30 at grade twelve to
44 at grade eight. In mathematics, the percentages ranged from 33 percent (at grade
twelve) to 46 percent (at grade eight).
Respondents indicated that 37 percent of grade-four LEP students, 27 percent of
grade-eight LEP students, and 22 percent of grade-twelve LEP students used
accommodations and adaptations in achievement testing in their schools.
The operational criteria used in NAEP from 1990 to 1996 indicated that LEP
students enrolled in schools where English is the primary language of instruction for
two or more years were to be included in the assessment. At least 85 percent of
fourth-grade students, 67 percent of eighth-grade students, and 83 percent of
twelfth-grade students had been enrolled for two or more years in schools where
English was the primary language. Historically, NAEP inclusion rates for LEP
students have been below the ideal minimums suggested by questionnaire results.
As was the case for students with disabilities, comparisons of questionnaire results
with assessment inclusion rates for LEP students suggest that: 1) increases in the
percentage of LEP students are not likely to result solely from revisions to inclusion
criteria that do not also involve the provision of accommodations; and 2) further
modest improvements in inclusion might still be possible if the list of permitted
accommodations and adaptations can be expanded.
Analyses of inclusion rates by the length of time students were enrolled in schools
where English is the primary language of instruction provided some evidence that,
when implemented without the provision of accommodations and adaptations, the
revised criteria actually resulted in less inclusion among LEP students than did the
original criteria. This evidence was strongest at grade four.
Under the revised criteria, all students receiving academic instruction in English for
three or more years were to be included in NAEP. Analyses based on questionnaire
responses as to the number of years students were receiving academic instruction in
English indicated that this ideal was not quite achieved. Inclusion rates among
students with three or more years of academic instruction in English were high, but
total inclusion was not achieved, even where accommodations and adaptations were
Some LEP students who do not usually receive accommodations in testing were
apparently provided accommodations in the NAEP assessment. The percentages of
LEP students in this category were small (10, 6, and 5 percent in grades four, eight,
and twelve, respectively).
Questionnaire results suggest that the procedural modifications made to NAEP had
their primary impact on inclusion rates at grades four and eight among students who
would be tested in their native language if this accommodation were available.
Participation rates for these students were higher when accommodations were
- Reese, C.M., Miller, K.E., Mazzeo, J., & Dossey, J.A. (1997). NAEP 1996 mathematics report card for the nation and
the states. (p. 67). Washington, DC: National Center for Education Statistics.
- National Academy of Education. (1993). The trial state assessment: Prospects and realities; the third report of the
national academy of education panel on the evaluation of the NAEP 1992 trial state assessment, Stanford, CA: Author.
- Olson, J.F., & Goldstein, A.A. (1996). Increasing the inclusion of students with disabilities and limited English
proficient students in NAEP. Focus on NAEP. 2(1). Washington, DC: National Center for Education Statistics.
- IRT analyses provides a common scale on which performance can be compared across groups such as those defined by
grade and characteristics, including gender and race/ethnicity.
Download the complete report in a PDF file for viewing and printing.
NCES 2000-473 Ordering information
U.S. Department of Education. Office of Educational Research and Improvement. National Center for Education
Statistics. Increasing the Participation of Special Needs Students in NAEP: A Report on 1996 NAEP Research Activities,
NCES 2000-473, by J. Mazzeo, J. E. Carlson, K. E. Voelkl, & A. D. Lutkus. Washington, DC: 2000.
Last updated 14 March 2001 (RH)