International Mathematics and Science Assessments: What Have We Learned?

January 1992

Executive Summary

The changing world economic order, foreshadowing new demands on the labor force and workplace, highlights the larger international context within which American education must be viewed. In January 1990, President Bush and the Nation's Governors recognized these evolving needs and established a specific goal for mathematics and science education—two subject areas critical to successful competition among highly technological societies: "By the year 2000, U.S. students wil be first in the world in science and mathematics achievement." To measure progress toward this objective, there is increasing interest in the periodic international assessments of student performance in mathematics and science.

Over the past quarter century, there have been five major international studies of science and mathematics achievement at the elementary, middle, and secondary school levels. The studies have been conducted under the auspices of two different nongovernmental research consortia. More than 30 countries have participated in at least one of the surveys. The United States has been involved in every one. A great variety of findings have resulted from this work, and these studies represent valuable contributions to the ways in which schooling inputs and outcomes are understood. The research has challenged participating countries to examine the structure, practices, and curricula of their educational systems, and as a consequence, to envision the possibility of rethinking curriculum content and the ways in which students are taught.

This report provides a description of the international assessments and some of their findings, and addresses issues surrounding the collection and analysis of these data. Further, it offers suggestions about ways in which new data collection standards could improve the quality of the surveys and the utility of the reports in the future.

Three Mathematics Surveys

The First International Mathematics Study, conducted in the 1960s, involved 13-year-old students from 10 countries and students in their last year of secondary school from 10 countries.

The Second International Mathematics Study, performed in the early 1980s, involved 13-year-old students from 18 countries and students in their last year of secondary school from 13 countries.

The First International Assessment of Educational Progress, carried out in 1988, involved 13-year-olds from six countries.

Three Science Surveys

The First International Science Study, conducted between 1966 and 1973, involved 10-year-old students from 16 countries, 14-year-old students from 18 countries, and students in their last year of secondary school from 18 countries.

The Second International Science Study, performed between 1983 and 1986, involved 10-year-old students from 15 countries, 14-year-old students from 17 countries, and students in their last year of secondary school from 13 countries.

The First International Assessment of Educational Progress, carried out in 1988, involved 13-year-olds from six countries.

The evidence suggests, in general, that students from the United States have fared quite poorly on these assessments, with their scores lagging behind those of students from other developed countries. This finding is based largely on analyses of mean achievement scores and related rankings of countries participating in each survey. Understanding that large-scale surveys pose a variety of analytical constraints and profit when complemented by more intensive case studies of particular findings, the international assessments do not explain why students from some countries perform better than their American counterparts. In fact, regular and sytematic patterns of differences are absent. For example, while students from some countries may do better on some or most of the achievement tests than students from other countries, the findings are age-group and subject-matter specific. Hence, they are very difficult to generalize since they are not the product of a single set of related, overriding school or institutional factors. Even so, across the studies certain trends appear to be clear:

The more students are taught, the more they learn, and the better they perform on the tests. There are significant differences in the content of instruction among countries at common levels of schooling.

Use of a differentiated curriculum based on tracking is negatively associated with student performance on the international assessments and also reduces opportunities for some students to be exposed to more advanced curriculum.

The school affects learning in some subject areas more than in others.

Countries committed to keeping students enrolled in secondary school score less well on the international surveys, but they spread more knowledge across a larger population. Japan is an exception. Even with high retention rates at the secondary level, Japanese students perform very well on the mathematics and science achievement surveys.

Generally the "best students" in the United States do less well on the international surveys when compared with the "best students" from other countries.

A number of technical considerations inhibit generalizing many other findings. The surveys have not achieved high degrees of statistical reliability across the age groups sampled and among all of the participating countries. Thus, from a statistical point of view, there is considerable uncertainty as to the magnitude of measured differences in achievement. Inconsistencies in sample design and sampling procedures, the nature of the samples and their outcomes, and other problems have undermined data quality. But despite their shortcomings, international achievement surveys now provide valued ways of documenting differences and investigating issues in student performance cross-nationally. The challenge in the future will be to make certain that these surveys meet quality technical standards.

From all indications, the various international testing authorities and consortia are moving expeditiously toward improving the quality of the surveys and upgrading their statistical reliability before the next rounds of international mathematics and science studies. Among the important tasks that lie ahead are strengthening the comparability of samples from country to country and developing new ways of reporting international achievement scores that will meet a variety of requirements and interests. It is noted that a considerable need also exists for small-scale case studies. These studies achieve in depth what they lack in breadth and help researchers understand the circumstances contributing to differences in performance among systems of education.

The report concludes by suggesting that there is a need for more deliberate consideration of policy concerns in the design of international assessments. This, in turn, may provide opportunities for policymakers and education practitioners to apply what is learned about cross-national differences in achievement to curriculum development and programming.

Download/view the full report in a PDF file.(5,606K)

Help with PDF files

For more information about the content of this report, contact Eugene Owen at Eugene.Owen@ed.gov.