CHAPTER 1: ACHIEVEMENT
U.S. fourth graders score above average in both mathematics and science compared with the 26 nations in the TIMSS fourth-grade assessment.
U.S. students' international standing is stronger at the fourth grade than it is at the eighth grade in both mathematics and science.
U.S. students' international standing is stronger in science than it is in mathematics at both the fourth and eighth grades.
In mathematics, 9 percent of U.S. fourth graders would rank among the world's top 10 percent. In science, 16 percent of U.S. fourth graders would rank among the world's top 10 percent.
There is no significant gender gap in fourth-grade mathematics achievement. However, in some content areas of fourth-grade science, U.S. boys outperform U.S. girls.
In the past, the mathematics and science achievement of U.S. students has been a cause for concern. International studies of these subjects conducted over the past 30 years show that our students have not performed as well as we might expect in comparison with their peers in other nations, especially in mathematics.
The patterns of academic achievement, however, vary widely. In a recent IEA study of reading literacy,\5\ U.S. fourth graders were second only to Finland, and U.S. eighth graders ranked among the top nations. Data from the eighth-grade TIMSS study,\6\ released in November 1996, showed that U.S. eighth graders score above the international average of the 41 TIMSS countries in science but below the international average in mathematics.
Compared with their international counterparts, U.S. fourth graders perform above the international average of the 26 TIMSS countries in both mathematics and science. In science, our students are outperformed by only one country--Korea.
Tempting though it may be, it is not correct to report U.S. scores by rank alone, as would be the case if one were to say U.S. fourth graders are "number x in the world in mathematics." This is because the process of calculating each country's score from the sample of students who took the test produces only an estimate of the country's score, not the true score itself. This estimate has a margin of error which is expressed as a "plus or minus" interval around the estimated score. In TIMSS, we can say with 95 percent confidence that comparisons of other countries' scores to those of the U.S. are accurate plus or minus about 15 points, depending on the design of the sample in the other countries. Comparisons of the U.S. with the international average are accurate plus or minus about 6 points. (Appendix 3 contains a list of country means and standard errors.) Because the precise score cannot be determined with perfect accuracy, to fairly compare the U.S. with other countries, nations have been grouped into broad bands according to whether their performance is higher than, not significantly different from, or lower than the U.S.
In mathematics, fourth-grade students in 7 countries outperform our fourth graders (Singapore, Korea, Japan, Hong Kong, the Netherlands, the Czech Republic, and Austria). Students in 6 countries are not significantly different from ours (Slovenia, Ireland, Hungary, Australia, Canada, and Israel). U.S. fourth graders outperform their counterparts in 12 nations (Latvia, Scotland, England, Cyprus, Norway, New Zealand, Greece, Thailand, Portugal, Iceland, Islamic Republic of Iran, and Kuwait).
In science, students in only one country--Korea--outperform U.S. fourth graders. Students in 5 countries are not significantly different than ours (Japan, Austria, Australia, the Netherlands, and the Czech Republic), and U.S. fourth graders outperform their counterparts in 19 nations (England, Canada, Singapore, Slovenia, Ireland, Scotland, Hong Kong, Hungary, New Zealand, Norway, Latvia, Israel, Iceland, Greece, Portugal, Cyprus, Thailand, Islamic Republic of Iran, and Kuwait).
TIMSS is a fair comparison of achievement for several reasons. First, the test was jointly developed and carefully reviewed by the participating countries to ensure that the items reflected curriculum topics considered important in all countries, and did not over-emphasize the curriculum content taught in only a few. Second, international monitors carefully reviewed nations' adherence to guidelines to ensure that significant numbers of students were not excluded from the testing process for any reason. Those nations that did exclude more than 10 percent of their students are clearly noted in this and other TIMSS reports. Therefore, we can be sure that the TIMSS scores in this report are a fair comparison of virtually all students at the appropriate grade in the various countries.
TIMSS required participating nations to adhere to extremely high technical standards at all stages of participation in the project. Of the 26 nations that participated at the fourth grade, 17 met or came close to meeting all technical standards for the study. The remaining 9 nations, however, experienced difficulties of various types. In some countries, the problems arose because a sizable proportion of schools, teachers, or students declined to participate. In others, the selection of schools or classrooms was not carried out according to international specifications. In still others, students were slightly older than the international target age. The names of those nations in which major difficulties arose are shown in parentheses in the figures in this report, and Appendix 4 describes these problems encountered. Because of the problems, the same amount of confidence cannot be attached to the scores of these 9 countries as to the other 17.
When the international average is calculated only from the 17 countries that met the international specifications, the mathematics and science scores of U.S. fourth graders are still above the international average for these 17 countries. Figure 3 shows our standing in comparison with these 17 nations.
What do the test scores mean? Due to the complex nature of the design, scoring, and analysis of the TIMSS test, a score of 600 does not mean either 600 items, or 60 percent, correct. Instead, this score indicates where the performance would fall if all fourth-grade scores were arranged along a scale running from 0 to 1,000.
In mathematics, the international average score was 529. A score of 658 or above would put a student in the top ten percent of all mathematics students in the 26 TIMSS countries, and a score of 601 would put a student in the top quarter.
In science, the international average score was 524. A score of 660 or higher would put a student in the top ten percent of all science students, and a score of 607 would put a student in the top quarter.
We can say with confidence that 5 countries outperform the U.S. in mathematics at the fourth grade (Singapore, Korea, Japan, Hong Kong, and the Czech Republic). The Netherlands and Austria also outperform us in fourth-grade mathematics, but due to deviations in their administration of TIMSS, we have less confidence in their scores. Only one nation outperforms the U.S. in science (Korea). Therefore, at the fourth grade, only Korea outperforms the U.S. in both mathematics and science. Four nations that outperform us in mathematics are not significantly different from us in science (Japan, Austria, the Netherlands, and the Czech Republic). Two nations that outperform the U.S. in mathematics score lower than the U.S. in science (Singapore and Hong Kong).
We can say with confidence that in both mathematics and science, U.S. fourth graders outperform their counterparts in 9 countries. These are:
- Seven European countries--Iceland, England, Scotland, Norway, Greece, Cyprus, and Portugal.
- Islamic Republic of Iran and New Zealand.
U.S. fourth graders also outperform Latvia, Kuwait, and Thailand in both subjects, but due to deviations in their administration of TIMSS, we have less confidence in their scores.
The "Group of Seven" or G-7 countries are major U.S. economic and political allies. The other six nations in this group are Canada, France, Germany, Italy, Japan, and the United Kingdom. Italy did not administer the TIMSS test, and France and Germany did not participate in the testing of fourth-grade students. Thus the U.S. can only be compared with the United Kingdom, Canada, and Japan. The United Kingdom is made up of England, Scotland, Northern Ireland, and Wales; however, the latter two did not participate in TIMSS. England and Scotland both have the same international standing in comparison with the U.S. Therefore, in this section, we describe our standing in relation to England.
Except for Japanese scores in mathematics, U.S. fourth graders' mathematics and science scores are similar to or higher than those of the other three participating G-7 nations. Japanese fourth graders outperform their U.S. counterparts in mathematics. U.S. fourth graders' mathematics scores are not significantly different from those of Canada and are higher than those in England. In science, U.S. fourth graders' scores are not significantly different than those of Japan and are significantly higher than those of Canada and England.
Comparisons of averages tell us how typical students perform, but they do not tell us about the performance of our nation's best students, including those who are likely to continue to study mathematics and science in secondary school and eventually become the next generation of mathematicians, scientists, doctors, and engineers. If an international talent search were to select the top ten percent of all fourth-grade students in the 26 TIMSS countries combined, what percentage of U.S. students would be included?
In mathematics, 9 percent of U.S. fourth graders are in the world's top ten percent. This is well below the 39 percent of Singaporean students, 26 percent of Korean students, and 23 percent of Japanese students who would be chosen in the international mathematics talent search. In science, 16 percent of U.S. fourth graders would rank among the world's top ten percent. No country has significantly more of their students in the top ten percent, and 21 nations have a smaller percentage. Figure 4 shows results for selected countries.
If the international talent search were to lower its standards so as to choose the top half of all students in the 26 TIMSS countries, in mathematics 56 percent of U.S. fourth graders would be included. This compares with 85 percent in Korea, 82 percent in Singapore, and 79 percent in Japan. In science, 63 percent of U.S. fourth graders would be in the top half of the students in the TIMSS countries, compared with 81 percent of students in Korea and 68 percent in Japan.
Representing student achievement in mathematics and science as a total score is a useful way to summarize achievement. Mathematics and science, however, contain very different content areas, which are emphasized and sequenced differently in curricula around the world. Based on these national priorities, some content areas are emphasized more than others at a particular grade level.
The TIMSS fourth-grade mathematics test included sets of items designed to sample students' ability to do work in the following areas:
- Whole Numbers (place value; ordering; comparing; problem-solving using addition, subtraction, and multiplication).
- Fractions and Proportionality (recognition and work with fractions and decimals; word problems).
- Measurement, Estimation, and Number Sense (common measures of size, time, temperature; rounding and estimation).
- Data Representation, Analysis, and Probability (use of data in charts, tables, and graphs; basic concepts underlying probability).
- Geometry (visualization of two- and three-dimensional forms; basic terms and properties; equivalence of figures; coordinate points on grids).
- Patterns, Relations, and Functions (patterns of numbers and shapes; representation of simple numerical situations; relationships of sequences of numbers).
In five of the six TIMSS mathematics content areas, the scores of U.S. fourth graders are above the international averages for those content areas. U.S. fourth-graders' performance is above the international average in whole numbers; fractions and proportionality; data representation, analysis and probability; geometry; and patterns, relations, and functions. In only one content area is the U.S. average below the international average--measurement, estimation and number sense. Figure 5a/5b shows these results.
In science, the TIMSS fourth-grade test sampled students' ability to do work in the following subjects:
- Earth Science (earth features; earth processes; earth in the solar system).
- Life Science (structure; diversity; classification; processes; cycles; and interactions of plants and animals).
- Physical Science (matter; energy and physical processes; forces and motion; physical and chemical changes).
- Environmental Issues and the Nature of Science (environmental and resource issues; nature of scientific knowledge; interaction of science and technology).
U.S. fourth graders score above the international average in all four science content areas. In three of these content areas--earth science; life science; and environmental issues, and the nature of science--U.S. fourth-grade students are outperformed by one or two other nations. In physical science, U.S. students are outperformed by 5 other nations. Figure 6a/6b shows these results.
Policy makers in the U.S. and other countries have made great efforts in recent years to make mathematics and science more accessible to girls and to encourage gender equity in these subjects. Overall, at the fourth grade, more TIMSS countries have gender equity in mathematics than in science.
The U.S. is one of 22 TIMSS nations in which there is no significant gender gap in fourth-grade mathematics achievement. In the science overall score, the U.S. is one of ten countries where a gender gap exists. Examining boys' and girls' scores in the various science content areas, U.S. boys significantly outperform U.S. girls in the content areas of earth science and physical science. There is no significant difference between U.S. boys' and girls' scores in life science and in environmental issues and the nature of science.
International comparisons over time are difficult. The first international studies of mathematics and science achievement were conducted in the 1960s, and there have been other assessments in each subject since that time. However, most assessments have focused on middle-school students and students in the final year of high school. Assessments of students in the elementary school grades have been conducted less frequently. Prior to TIMSS, only one international assessment of elementary-school children was undertaken in mathematics, although there were three prior assessments in science.
However, each assessment was done a little differently. A different set of nations participated, different topics in mathematics and science were included in the tests, the age and type of students sampled in each country changed slightly, and even the borders and names of some of the nations have changed. These and other factors complicate comparisons over time and require that any conclusions that are drawn be necessarily tentative.
Among the various international studies conducted over the past 30 years, only the International Assessment of Educational Progress (IAEP) tested the mathematics achievement of elementary-school students. In the 1991 IAEP assessment of 9-year-olds in 14 nations,\7\ U.S. students scored below the international average in mathematics. However, as we have seen, in the 1995 TIMSS study reported here, U.S. fourth graders scored above the international average of 26 nations in this subject. Comparisons over time are difficult, so caution should be exercised in assuming there has been significant improvement in our fourth graders' international standing in mathematics, but it is a possibility.
Three previous international science assessments of elementary-school students were conducted in the 1960s, 1980s, and early 1990s. The U.S. scored above the international average in two of these three studies. In the other study, the U.S. was not different than the international average. Moreover, in all three previous studies, only a few nations outperformed the U.S. In the 1960s study, one nation out of 11 (Japan); in the 1980s study, 5 nations out of 14 (Japan, Korea, Finland, Sweden, and Hungary)\8\, and in the 1991 study, one nation out of 13 (Korea) outperformed the U.S.\9\ Taken together with the TIMSS findings reported here, it appears that U.S. students in the middle years of elementary school perform reasonably well in science in comparison with their peers in other nations. It is not clear whether this relative international standing has changed over time.
In both mathematics and science, our international standing is higher at fourth grade than it is at eighth grade. Figure 7 provides a quick overview of mathematics and science performance at each grade level, in comparison with all of the countries participating at each grade level.
In mathematics, our fourth-grade students score above the international average, while our eighth-grade students score below the international average. In science, U.S. students score above the international average at both grade levels. However, only one nation outperforms us at the fourth grade, while 9 nations outperform us at the eighth grade.
This pattern in relative international standing is also evident when one takes into account the fact that 41 nations participated in eighth-grade TIMSS, whereas 26 nations participated in the fourth-grade study. Comparisons of U.S. total performance among the 26 nations that participated in both the fourth-grade and eighth-grade TIMSS studies confirm this observation. Among these 26 nations in mathematics, U.S. fourth graders score above the international average and are outperformed by 7 nations, whereas U.S. eighth graders score below the international average and are outperformed by 13 nations. In science, U.S. fourth graders score above the international average and are outperformed by only one other nation. U.S. eighth graders score not significantly different from the international average and are outperformed by 8 other nations.
While the U.S. international standing is lower at fourth grade than at eighth grade, most other countries (19 in both mathematics and science) have a similar standing relative to the international average at both grade levels. Five countries have a lower relative standing at eighth grade than at fourth grade in one subject, and 4 countries (the U.S., Scotland, Ireland, and Latvia) have a lower relative standing at eighth grade in both subjects. Only one country--the U.S. in mathematics--falls from above the international average at fourth grade to below the international average at eighth grade.
Another way of looking at the U.S. performance at fourth and eighth grade is to see how many countries compare more favorably to the U.S. at eighth grade than they did at fourth grade. Of the 25 countries that participate at both grade levels, all perform as well or better relative to the U.S. at eighth grade than at fourth grade. That is, no country compares less favorably to the U.S. in eighth grade than it does in fourth grade, and most compare more favorably. In both subjects, most of the countries (5 out of 6 in mathematics and 4 out of 5 in science) with average scores similar to the U.S. in fourth grade have scores in eighth grade that are significantly higher than the U.S. Likewise, many of the countries (8 of 12 in mathematics and 9 of 19 in science) whose scores are below the U.S. in fourth grade have eighth-grade scores that are similar to the U.S., and in science, 3 countries (Singapore, Slovenia, and Hungary) have fourth-grade scores below the U.S. and eighth-grade scores above the U.S.
The picture for the content areas of mathematics and science is somewhat more complicated. Figure 7 displays the results by content area. In mathematics, the fourth-grade and eighth-grade tests have only two content areas in which scores are reported for both grade levels (data representation, analysis, and probability; and geometry). U.S. students' scores in data representation, analysis and probability are significantly higher than the international average at both grade levels. U.S. students' scores in geometry are above the international average at the fourth grade and below the average at the eighth grade.
With regard to the other mathematics content areas, U.S. fourth graders exceed the international average in three content areas (whole numbers; fractions and proportionality; and patterns, relations, and functions) but are below it in one (measurement, estimation, and number sense). Eighth graders are not different from the international average in two content areas (fractions and number sense; and algebra) and are below the international average in two areas (measurement and proportionality).
In science, the fourth-grade and eighth-grade tests have three content areas for which scores are reported at both grade levels. U.S. fourth-graders and eighth-graders' scores are higher than the international average in all three of these content areas (earth science; life science; and environmental issues and the nature of science). U.S. fourth-graders' scores are above the international average in physical science, whereas eighth-graders' scores are not different than the international average in physics and chemistry.
The foregoing discussion of TIMSS achievement findings highlights three important patterns:
- U.S. fourth graders are above the international average in both mathematics and science.
- The international standing of U.S. fourth graders is stronger than that of eighth graders in both subjects.
- U.S. students perform better in science than in mathematics at both the fourth and eighth grades in comparison with their international counterparts.
The next chapter explores the initial evidence from TIMSS concerning various factors that may contribute to these patterns.