• U.S. eighth graders score below average in mathematics achievement and above average in science achievement, compared to the 41 nations in the TIMSS assessment.
  
  
• In mathematics, our eighth-grade students' international standing is stronger in Algebra and Fractions than in Geometry and Measurement.
  
  
• In science, our eighth graders' international standing is stronger in Earth Science, Life Science, and Environmental Issues than in Chemistry and Physics.
  
  
• The U.S. is one of 11 TIMSS nations in which there is no significant gender gap in eighth-grade math and science achievement.

In the past, the mathematics and science achievement of U.S. students has caused nation-wide cries for improvement. Various international studies of these subjects conducted over the past thirty years have shown that our eighth graders have not performed as well as we expect, in comparison to their peers in other nations. U.S. students are not weak in all subjects, however. In a recent IEA study of reading literacy3, U.S. eighth graders were among the best in the world. Indeed, TIMSS shows that U.S. eighth grade students also scored better than the average of the 41 participating countries in science. The results in mathematics, however, put our nation below average compared to the other nations.



HOW WELL DO U.S. STUDENTS DO?

Compared to their international counterparts, U.S. students are somewhat below the international average of 41 TIMSS countries in mathematics. In science, our students are somewhat above the international average. Figure 1 and Figure 2 show how U.S. students perform in these subjects.

Tempting as it may be, it is not correct to report U.S. scores by rank alone, as would be the case if one were to say the U.S. is "number x in mathematics out of the 41 TIMSS countries." This is because the process of estimating each country's score from the sample of students who took the test produces only an estimate of the range within which the country's real score lies. This margin of error is expressed as a "plus or minus" interval around the estimated score.

In TIMSS, we can say with 95 percent confidence that comparisons of other countries to the U.S. are accurate plus or minus about 20 points, depending on the size and design of the sample in the other countries. Comparisons of the U.S. to the international average are accurate plus or minus about 10 points. (Appendix 3 contains a list of standard errors). Because the precise score cannot be determined with perfect accuracy, to fairly compare the U.S. to other countries, nations have been grouped into broad bands according to whether their performance is higher than, not significantly different from, or lower than the U.S.

In mathematics, students in 20 countries outperform our eighth graders. Students in 13 countries are not significantly different than ours, and U.S. students outperform their counterparts in 7 nations. In science, students in 9 nations outperform U.S. eighth graders, performance in 16 other nations is not statistically different than ours, and we outperform another 15 nations.



SOME SPECIAL NOTES ON THE TEST SCORES

TIMSS required participating nations to adhere to extremely high technical standards at all stages of participation in the project. Many nations experienced some difficulty in this respect. In two nations, difficulties in meeting the standards were so severe that international monitors decided that their data should not be included in the report, and so findings are reported only for the remaining 41 nations. Of the 41 nations, 25 met or came close to meeting all technical standards for the study. However, 16 nations experienced difficulties of various types. In some countries, these difficulties arose because a large proportion of schools, teachers, or students declined to take the test. In others, the selection of schools or classrooms was not carried out according to international plan. In still others, students were slightly older than the international target age. The names of those nations in which major difficulties arose are shown in parentheses in the figures in this report, and Appendix 4 describes any deviations from international specifications that occurred. It should be kept in mind that we cannot have the same amount of confidence in the scores of the 16 nations in which major difficulties arose.

If the international average is calculated only from the 25 countries in which no major difficulties arose in carrying out the international specifications, the U.S. mathematics score is still below the international average. In science, however, our score is no longer significantly different from the average of the 25 nations. Our comparative position in science becomes lower because many of the countries who are removed from consideration are those that we outperformed. Figure 3 shows our mathematics and science standing in comparison to these 25 nations, and the types of anomalies that occurred in the other 16 countries. The difference in U.S. standing between Figure 3 and the previous figures demonstrates that the selection of countries against which the U.S. is compared can change our international standing.

Which comparison should we emphasize as TIMSS' main finding the comparison to 25 countries, or to 41? NCES has chosen as the primary finding our standing with respect to 41 countries because the international TIMSS reports present the results in terms of all 41 nations.

What do the test scores mean? Due to the complex nature of the TIMSS test design, scoring, and analysis, a score of 600 does not mean either 600 items, or 60 percent correct. One can interpret the scores by considering where they fall along the range of scores from 0 to 1000 of other eighth-grade students who took the test. In mathematics, a score of 656 would put a student in the top 10 percent of all students in the 41 TIMSS countries, and a score of 587 would put a student in the top 25 percent. In mathematics, 509 was the average student score. In science, a score of 655 would put a student in the top 10 percent, a score of 592 would put a student in the top 25 percent, and 522 was the average student score.



WHICH COUNTRIES OUTPERFORM THE U.S. IN BOTH SUBJECTS?

We can say with confidence that five nations outperformed us in both mathematics and science. They are:

• Three Asian nations - Singapore, Korea, and Japan.
• Two Central European nations - Czech Republic and Hungary.

We can say with confidence that the U.S. outperformed four countries in both mathematics and science:

• Three European countries - Lithuania, Cyprus, and Portugal.
• One Middle Eastern country - Iran.

The U.S. also outperformed Kuwait, Colombia, and South Africa in both subjects, but due to deviations in their administration of TIMSS, we have less confidence in their scores. These seven countries were the only ones that we outperformed in mathematics, and they were also among the 15 countries that we outperformed in science.



HOW DO WE COMPARE TO OUR MAJOR ECONOMIC PARTNERS?

The "Group of Seven" or G-7 countries are major U.S. economic and political allies. The other six nations in this group are the United Kingdom, France, Germany, Canada, Japan, and Italy. Italy did not administer the TIMSS test, so the U.S. can only be compared to the remaining five. The United Kingdom includes England, Scotland, Northern Ireland, and Wales. Northern Ireland and Wales did not participate in TIMSS, and England and Scotland both have the same international standing in comparison to the U.S. Therefore, in this section, we describe our standing in relation to England.

In mathematics, Japan, France, and Canada outperform the U.S., while our scores are not significantly different from those of England and Germany. In science, we score lower than Japan; were not significantly different than England, Canada, and Germany; and score higher than France. Considering our standing in relation to these five major economic partners, it can be said that the U.S. is in the bottom half in mathematics, and about the middle in science.

Among the G-7 countries, Germany is the only nation which appears in parentheses, indicating problems in the implementation of the international guidelines for carrying out the study. In Germany, the problem was a discrepancy in the age of the students tested. Because German children start school somewhat later than children in other countries, the average age of the German eighth-graders who took the TIMSS test was about four months older than the international target age. Some would say that this means that other nations' eighth graders should be compared with Germany's seventh graders for a better age comparison. However, this provides a less-than-ideal grade comparison.

In a grade-based comparison, there is no significant difference between German and U.S. eighth graders. If we were to approximate an age-based comparison by matching the scores of our eighth graders to those of German seventh graders, our eighth graders do significantly better. Both comparisons are useful because most experts believe that achievement is based partly on cognitive maturation which comes with age, and partly on years of study which come with grade in school.



HOW FAR BEHIND THE TOP COUNTRIES ARE WE?

Particularly in mathematics, our students are far behind Singapore and Japan which are among the top-scoring nations in the world in both math and science. One way to compare two nations' scores is by considering their comparative standing with relation to the international percentiles. In mathematics, the scores of our very best U.S. eighth graders, who perform at the 95th percentile for our nation, are not significantly different than the scores of average eighth graders in Singapore, who perform at their nation's 50th percentile. In comparison to Japan, the scores of our best students, who are at the 95th percentile for our nation, are significantly below the scores of the top quarter of Japanese students, who perform at their nation's 75th percentile.

In science, the gap is not so large. Students at the U.S. 95th percentile are significantly better than students at the 75th percentile in Singapore. In comparison to Japan, there is no significant difference between U.S. and Japanese students at the 95th percentile.

Another way to estimate distance between the U.S. and top scoring countries is to use the difference between our seventh and our eighth graders as a unit of measure. In mathematics, the difference between our seventh and eighth graders' scores was 24 points. The difference between the scores of eighth graders in the U.S. and in Singapore was 143 points. This means that the difference in eighth-grade mathematics performance between the two countries is almost six times the difference between U.S. seventh and eighth graders. The difference between U.S. and Japanese eighth graders' mathematics performance is about four times this difference.

In science, the gap is smaller, but still substantial. The difference between U.S. seventh and U.S. eighth graders' scores is 26 points. The difference between the scores of the U.S. and Singapore was 73 points. The difference in science performance between eighth graders in the U.S. and Singapore is almost three times the difference between our seventh and eighth graders. The difference between U.S. and Japanese eighth graders' science performance is almost one and a half times this difference.



HOW DO OUR BEST STUDENTS COMPARE WITH OTHERS' BEST?

Comparisons of averages tell us how typical students perform, but they do not tell us about the performance of our nation's best students - those who are likely to become the next generation of mathematicians, scientists, doctors, and engineers. If an international talent search were to select the top ten percent of all students in the 41 TIMSS countries combined, what percentage of U.S. students would be included?

In mathematics, 5 percent of U.S. eighth graders would be selected. High-scoring nations would have more of their students represented in the "international top ten percent." Figure 4 shows that 45 percent of all Singaporean students and 32 percent of all Japanese students would be chosen in the international talent search in mathematics. In science, 13 percent of U.S. students would be selected, in comparison to 31 percent of Singaporean students and 18 percent of Japanese students.

If the international talent search were to lower its standards considerably to choose the top half of all students in the 41 TIMSS countries, 94 percent of eighth graders in Singapore and 83 percent in Japan would be selected in mathematics, compared to 45 percent of eighth graders in the U.S. In science, 82 percent of the students in Singapore and 71 percent of students in Japan would be selected, compared to 55 percent in the U.S.



HOW DOES THE U.S. MATHEMATICS AND SCIENCE
GENDER GAP COMPARE TO OTHER COUNTRIES'?

In the U.S. and in other countries, policy makers have made great efforts to make math and science more accessible to girls, and to encourage gender equity in these subjects. More TIMSS countries have achieved gender equity in their students' scores in mathematics than in science. The U.S. is one of 11 TIMSS nations in which there is no significant gender gap in eighth-grade mathematics and science achievement. The U.S. was one of 33 countries in which there was no statistically significant difference between the performance of eighth-grade boys and girls in mathematics. In science, we were one of 11 nations with no statistically significant difference. All 11 nations with no significant different in science also demonstrated no difference in mathematics. They are the United States, Singapore, the Russian Federation, Thailand, Australia, Ireland, Romania, Flemish Belgium, Cyprus, Columbia, and South Africa.



HOW DO WE SCORE IN THE DIFFERENT CONTENT AREAS OF MATHEMATICS AND SCIENCE?

Representing student achievement in mathematics and science as a total score is a useful way to summarize achievement. However, mathematics and science contain different content areas, which are emphasized and sequenced differently in curricula around the world. Based on these national priorities, in each country, some content areas have been studied more than others at a particular grade level.

The TIMSS eighth-grade mathematics test included sets of items designed to sample students' ability to do work in the following areas:

• Algebra (patterns, relations, expressions, equations).
• Data Representation, Analysis, and Probability (representation and analysis of data using charts and graphs involving uncertainty and probability).
• Fractions and Number Sense (fractions, decimals, percentages, estimation and rounding).
• Geometry (visualization and properties of geometric figures, including symmetry, congruence, and similarity).
• Measurement (units of length, weight, time, area, volume, and interpretation of measurement scales).
• Proportionality (proportionality and ratios).

Figure 5 shows that among these content areas, U.S. students' performance is at about the international average in Algebra; Data Representation, Analysis, and Probability; and Fractions and Number Sense. Compared to other countries, we do less well in Geometry; Measurement; and Proportionality. Our weaker performance in these latter three topics may pull the overall U.S. score down to below average.

In science, the TIMSS eighth-grade test sampled students' ability to do work in the following subjects:

• Chemistry (classification of matter, chemical properties and transformations).
• Earth Science (earth features, earth processes, and the earth in the universe).
• Environmental Issues and the Nature of Science (environmental and resource issues, the nature of scientific knowledge, and the interaction of science and technology).
• Life Science (structure, diversity, classification, processes, cycles, and interactions of plants and animals).
• Physics (energy forms, physical transformations, force and motion, and physical properties of matter).

Figure 6 on pages shows our comparative standing in these content areas. The U.S. is among the top countries in the world in Environmental Issues and the Nature of Science, and we are also above the international average in Earth Science and Life Science. In Chemistry and Physics, our performance is not significantly different from the international average. Our better-than-average scores in Environmental Issues, Earth Science, and Life Science may pull our overall science score up to above average.



WHAT DID PRIOR STUDIES SHOW ABOUT HOW U.S. STATES
COMPARE TO OTHER COUNTRIES?

Comparison of U.S. states with other nations reminds us that not all U.S. school systems are alike, and that wide differences in achievement exist within our own nation. Some would say that comparisons of U.S. states and other nations are fair for two reasons. First, most U.S. states are larger either in size or population than many countries in the TIMSS study. For example, California is larger in size than Japan, Germany, or England. New Jersey has a larger population than Austria, Denmark, or Switzerland. A second reason that such comparisons are fair is that each U.S. state is responsible for its own education system, similar to the way in which most other TIMSS national governments are responsible for their own education system.

Future analyses may make possible such comparisons between U.S. states and the TIMSS nations. Efforts are now underway to create an experimental linkage between the TIMSS study and the mathematics and science portions of the National Assessment of Educational Progress (NAEP). This linkage will allow an estimation of how states would have performed on TIMSS if their students had taken the test. The results for eighth-grade mathematics and science will be announced in 1997.

Until those findings are released, however, we can look at the results of a similar linkage which was performed in 1991 for eighth-grade mathematics students' scores on NAEP and on the International Assessment of Educational Progress4. In that comparison, the mathematics scores of Iowa, North Dakota, and Minnesota were similar to top-scoring Taiwan and Korea. In contrast, Alabama, Louisiana, and Mississippi scored about the same as lowest-scoring Jordan. These findings underscore the considerable variation in achievement that exists among states within our own nation.



HAS U.S. INTERNATIONAL STANDING IMPROVED OVER TIME?

Results from the National Assessment of Education Progress show that our eighth-grade students' scores in math and science have improved somewhat in comparison to our own performance during the past decades. If our domestic performance over time is improving, how does this affect our international standing? It is possible that only U.S. achievement has improved over time, while achievement in other countries has not. Of course, it is also possible that improvements in the U.S. have been matched or exceeded by improvements in other countries.

International comparisons over time are difficult. The first international studies of math and science achievement were conducted in the 1960s, and there have been three previous assessments in each subject since that time. However, each assessment has been done differently. A different set of nations participated, different topics in math and science were included in the tests, the age and type of students sampled in each country changed slightly, and indeed even the borders and names of some of the nations have changed. Furthermore, the field of assessment has matured greatly over the past thirty years, rendering the methods of the then-revolutionary early studies crude by today's standards. These and other factors complicate comparisons over time, and require that any conclusions that are drawn be necessarily tentative.

In TIMSS mathematics, we have seen that our eighth-graders scored below the international average. This is basically the same relative international standing reported for U.S. thirteen-year-olds in the IEA First and Second International Mathematics Studies in the 1960s and 1980s, and the mathematics portion of the International Assessment of Educational Progress in the early 1990s5. Relative to their international counterparts, it is not likely that U.S. eighth-graders' standing in mathematics has improved significantly.

In the three previous international science assessments in the 1960s, 1980s, and early 1990s, the U.S. performed below the international average of thirteen or fourteen-year-olds. However in TIMSS, our students scored at or above the international average. Because comparisons over time are difficult, caution should be exercised in assuming there has been significant improvement in our international standing in science, but it is a possibility.

We have now examined what TIMSS tells us about what eighth-grade students have learned. Learning, of course, is closely related to what students are taught. Next we turn to an examination of how the U.S. mathematics and science curricula compare with those of other nations.