NAEP 1996 Science Report Card for the Nation and the States

May 1997

Authors: Christine Y. O'Sullivan, Clyde M. Reese, and John Mazzeo (Educational Testing Service)

Download the complete report in a PDF file for viewing and printing. 1,605K

Executive Summary^[1]

For more than a quarter of a century, the National Assessment of Educational Progress (NAEP) has reported to policy makers, educators, and the general public on the educational achievement of students in the United States. As the nation's only ongoing survey of students' educational progress, NAEP has become an important resource for obtaining information on what students know and can do.

The NAEP 1996 science assessment continues the mandate to evaluate and report the educational progress of students at grades 4, 8, and 12. The national results provided herein describe students' science achievement at each grade and within various subgroups of the general population. State-level results for grade 8 are presented for the 44 individual states and other jurisdictions that chose to participate in the 1996 state assessment and met the guidelines for participation. NAEP national and state data assess the performance of students in both public and nonpublic schools.

The NAEP 1996 Science Framework

The science assessment was crafted to measure the content and skills specifications described in the science framework for the 1996 National Assessment of Educational Progress. Two organizing concepts underlie the science framework. First, according to the framework, scientific knowledge should be structured so as to make factual information meaningful. The way in which knowledge is structured should be influenced by the context in which the knowledge is being presented. Second, science performance depends on knowledge of facts, the ability to integrate this knowledge into larger constructs, and the capacity to use the tools, procedures, and reasoning processes of science to develop an increased understanding of the natural world. Thus, the framework called for the NAEP 1996 science assessment to include the following:

Multiple-choice questions that assess students' knowledge of important facts and concepts and that probe their analytical reasoning skills;

Constructed-response questions that explore students' abilities to explain, integrate, apply, reason about, plan, design, evaluate, and communicate scientific information; and

Hands-on tasks that probe students' abilities to use materials to make observations, perform investigations, evaluate experimental results, and apply problem-solving skills.

The core of the science framework is organized along two dimensions. The first dimension divides science into three major fields: earth, physical, and life. The second dimension defines characteristic elements of knowing and doing science: conceptual understanding, scientific investigation, and practical reasoning. Each question in the assessment is categorized as measuring one of the elements of knowing and doing within one of the fields of science (e.g., scientific investigation in the context of earth science). The framework also contains two overarching domains the nature of science and the organizing themes of science. The nature of science encompasses the historical development of science and technology, the habits of mind that characterize science, and the methods of inquiry and problem solving. It also includes the nature of technology specifically, design issues involving the application of science to real-world problems and associated trade-offs or compromises. The themes of science include the notions of systems and their application in the scientific disciplines, models and their functioning in the development of scientific understanding, and patterns of change as they are exemplified in natural phenomena.

Student Achievement

Students' science performance is summarized on the NAEP science scales, which range from 0 to 300 at each grade. While the scale-score ranges are identical, the scales were derived independently at each grade. Therefore, average scale scores across grades cannot be compared. For example, equal scale scores on the grade 4 and grade 8 scales do not imply equal levels of science achievement. Within each of the three grades, scale scores for students ranged from about 105 for those scoring at the 10th percentile to about 192 for those performing at the 90th percentile.

It is possible to illustrate the level of achievement of students with a given scale score by identifying questions likely to be answered correctly by students with that scale score, a process known as "mapping." The position of the question on the scale represents the scale score attained by students who had at least a 65 percent probability of reaching a given score level on a constructed-response question or at least a 74 percent probability of correctly answering a multiple-choice question. Mapping questions onto the NAEP science scales can be used to illustrate the range of achievement of students at or near selected percentiles. For example, eighth graders at or near the 50th percentile were likely to correctly identify the source of acid rain. Put slightly differently, this question was answered correctly by at least 74 of every 100 students scoring at or above the 150 scale-score level.

Table: Sample Questions At
or Near Selected Percentiles

Major Findings for the Regions and States^[2]

NAEP data can be used to compare student performance of fourth-, eighth-, and twelfth-grade students attending schools in four different regions Northeast, Southeast, Central and West and state-level results for eighth-grade students.

For all three grades, students in the Central region had higher average scores than students in the Southeast region.

In 19 of the 44 participating jurisdictions^[3], the average scale score for public school eighth graders was higher than the national average, while 14 jurisdictions performed below this average. The remaining 11 jurisdictions performed at or around the national average.

Table: Summary of
Jurisdiction Performance Relative to the Nation for Grade 8 Public Schools

Major Findings for Student Subgroups

The NAEP 1996 science assessment reports national data on the basis of demographic subgroups, level of parental education, type of school attended, and participation in selected government programs.

Male and female students in grades 4 and 8 had similar scores. However in grade 12, male students had higher scores than female students.

White and Asian/Pacific Islander students had higher average scores than Black and Hispanic students at all three grades.

American Indian students had higher average scores than Black and Hispanic students, in grades 4 and 8. (The sample of American Indian students at grade 12 was too small to permit comparisons.)

In general, at all three grades higher levels of parental education were associated with higher levels of student performance.

At all three grades, students who attended nonpublic schools had higher average scores than those who attended public schools.

Fourth- and eighth-grade students receiving services supplied by Title I programs had lower scale scores than those who did not participate in Title I. (The sample for twelfth graders who participated was not large enough to permit a comparison.) Title I of the Elementary and Secondary Education Act provides funding to local educational agencies to meet the needs of children who are economically disadvantaged and who are performing below grade level.

At all three grades, students eligible for the free/reduced-price lunch program administered by the U.S. Department of Agriculture (USDA) scored lower than those not eligible. Eligibility for free/reduced-price lunches is determined by the USDA's Income Eligibility Guidelines. (Information on eligibility was lacking for 12 percent of fourth graders, 23 percent of eighth graders, and 21 percent of twelfth graders.)

Exploring a More Inclusive NAEP

An area in which the NAEP program continues to seek improvements is in the inclusion and appropriate assessment of two specific populations: students with disabilities (SD) and limited English proficient (LEP) students. The NAEP 1996 math and science assessments included supplemental samples of schools and students to enable the program to study the effects of revised inclusion rules on assessment results and to investigate the feasibility and impact of increasing the participation of students with disabilities and LEP students by offering assessment accommodations and adaptations.

Results from the grade 8 state NAEP science assessment indicated that the use of revised inclusion criteria, without the provision of accommodations, had little effect on the overall percentage of the total population assessed, or on the percentages of students with disabilities or LEP students assessed. There was some evidence from the national NAEP assessment that the provision of accommodations resulted in higher rates of participation for both groups of students.

The Executive Summary for this report was prepared by Alan Vanneman of the Education Statistics Services Institute.
In all discussions of differences in academic performance between subgroups, only statistically significant differences are reported. Such differences are unlikely to be due to chance factors.
Several states participated but failed to meet established participation guidelines for reporting results. See Appendix A for more complete information of jurisdictions' participation rates.

Download the complete report in a PDF file for viewing and printing. 1,605K

NCES 97-497 Ordering information

Last updated 4 April 2001 (RH)