MATHEMATICS AND SCIENCE GENERAL KNOWLEDGE

How well do young people understand the basic mathematics and science needed to function effectively in society? To answer this question, TIMSS developed assessments of mathematics and science general knowledge. These assessments were designed to determine students' general level of knowledge of fundamental scientific and mathematical concepts at the time they complete secondary education. These assessments were given to a random sample of all students at the grade set by their nation or program of studies as the end of their secondary schooling, regardless of whether or not they were currently taking mathematics or science at the time of the study.

Because the assessments were designed to examine how well students had acquired the mathematical and scientific skills and knowledge judged by an international committee of experts to be necessary for all citizens in their daily life, the questions asked of the students were not tied to school curriculum. Instead, they covered the students' knowledge of mathematical and scientific concepts, reasoning, and practical or "real world" applications. The results provide a glimpse of how well-prepared to function in the adult world are the graduates of the education system in the various TIMSS nations.

This report examines the performance of U.S. students relative to students in other participating countries. Tempting though it may be, reporting U.S. scores by rank alone would be incorrect. This is because the average scores reported for each country were based on a sample of students in each country, and are therefore estimates of the "true" scores that would have been achieved had all the eligible students participated in the assessments.

While many steps were taken to ensure that the samples were representative of the total population, each estimated score has a margin of error associated with it. The margin of error is expressed as a "plus or minus" interval around the estimated score, creating a range of scores within which the true score is likely to fall. Thus, while one score may be higher than another, if the difference between the two is small enough, it may fall within the margin of error and not be statistically significant. Because precise scores cannot be determined with perfect accuracy, to fairly compare the United States to other countries, nations have been grouped into broad bands according to whether their performance was significantly higher than, not significantly different from, or significantly lower than the United States.

In TIMSS, we can say with 95 percent confidence that comparisons of other countries' scores on the general knowledge assessments to those of the United States are accurate plus or minus about 12 to 36 points, depending on the size and design of the samples in other countries. Comparisons of the United States to the international average are accurate plus or minus about 7 points. (Table A2.1 in Appendix 2 contains a list of national averages and standard errors.)

MATHEMATICS GENERAL KNOWLEDGE ASSESSMENT

The mathematics general knowledge assessment consists of about 80 percent multiple choice items and 20 percent free-response items. Items were chosen based on their likelihood of arising in real-life situations and not on their connection to a particular curriculum. However, they can be described in terms of common mathematics curriculum topics, such as number sense, including fractions, percentages, and proportionality; algebraic sense; measurement and estimation; and data representation. On average, for the countries participating in the TIMSS assessment of mathematics general knowledge, these topics are typically covered in about the seventh grade.

How Well Do U.S. Twelfth Graders Do On The Mathematics General Knowledge Assessment?

On the mathematics portion of the general knowledge assessment, U.S. students scored below the international average, and among the lowest of the 21 countries. Figure 1 shows how U.S. students performed on the mathematics general knowledge assessment.

In mathematics general knowledge, students in the final year of secondary school in 14 countries scored above our twelfth graders (the Netherlands, Sweden, Denmark, Switzerland, Iceland, Norway, France, New Zealand, Australia, Canada, Austria, Slovenia, Germany, and Hungary). Students in 4 countries were not significantly different from ours (Italy, the Russian Federation, Lithuania, and the Czech Republic). Students in two countries (Cyprus and South Africa) performed significantly below students in the United States.

One explanation for our low performance that has been suggested in the past is that, because of our diverse population, there is a greater range of scores among U.S. students, and the difference between our lowest-scoring students and our typical student is greater than in many other countries. These low-scoring students, it has been argued, "bring down" the U.S. average. Available information suggests that this is not the case in TIMSS.^A

Looking at the distribution of scores can also illustrate the relatively low position of the United States in TIMSS. The entire distribution of U.S. scores is shifted downward from that of many of the high performing countries. For example, while a quarter of U.S. students scored 521 or higher, in many high-performing countries half or more of the students had scores that high. Furthermore, the scores of U.S. students at the 95th percentile were similar to those of students at the 75th percentile in some countries. (See Table A2.3 in Appendix 2 for percentiles for mathematics general knowledge; see Tables A2.4, A2.5, and A2.6 for percentiles for the other assessments.)

What Were Students Asked To Do On The Mathematics General Knowledge Assessment?

Mathematics general knowledge assessment items were designed to measure general knowledge and skills judged by an international committee of experts to be necessary for citizens in their daily life. Three examples of mathematics general knowledge assessment items are shown. Table A3.1 in Appendix 3 shows the percentage of students responding correctly to each example item in every country.

The item shown in Figure 2 requires students to use complex procedures to solve a percentage problem. Fifty-seven percent of U.S. twelfth graders responded correctly to this item. The international average was 64 percent correct. Some students who responded incorrectly chose "C," which is simply the difference of the two percentages, rather than correctly taking the product of the percentages.

The item shown in Figure 3 requires students to provide their response in an open-ended format. Eighty-five percent of U.S. students responded correctly on this item. The international average was 74 percent. Students needed to be able to read the line graph and use the labeled information on the vertical axis to provide the correct answer of 60 km/h as the car's maximum speed.

The item in Figure 4 requires students to use the dimensions of a figure to solve a problem. Thirty-two percent of U.S. twelfth graders answered this item correctly. The international average was 45 percent. Some students who responded incorrectly forgot to take into account in their calculations the sides of the box that are not visible in the diagram or the 25 centimeters of ribbon needed to tie a bow.

How Does U.S. Twelfth-Grade Students' Relative Performance In Mathematics Compare To That Of U.S. Eighth-Grade Students?

The group of countries participating in each phase of TIMSS differed. However, 20 of the 21 countries participating in the general knowledge assessments in the final year of secondary schooling also participated in the middle school portion of TIMSS. We can calculate an international average for mathematics achievement of students in these 20 countries both for eighth grade and for the final year of secondary schooling. (This international average will differ from that based on all countries participating in TIMSS at each grade level - 41 in eighth grade and 21 for the final year of secondary schooling. The average U.S. eighth grade mathematics performance is below the international average when the international average is based on all 41 countries participating in TIMSS at eighth grade, but is similar to the international average based on those 20 countries that also participated in the general knowledge assessments at the end of secondary schooling.) Table A5.1 in Appendix 5 shows the standing of each country relative to the 20-country international average for the two grade levels and whether that relative standing was different at the two grade levels. (See Table A5.2 in Appendix 5 for a similar comparison for the 12 countries that participated in TIMSS both at fourth grade and in the general mathematics knowledge assessment at the end of secondary school.)

The relative standing of U.S. students in mathematics was lower at twelfth grade than at eighth grade. About half the countries had a similar standing relative to the international average at both grade levels. The other half were about equally divided between those with a higher and a lower relative standing in the final year of secondary schooling than in eighth grade. The former group was composed of Nordic countries plus New Zealand, while the latter included countries from the former Communist Bloc and Australia, in addition to the United States.

Is There A Gender Gap In Mathematics General Knowledge At The Twelfth Grade?

In the United States and other countries, policy makers have made great efforts to make mathematics and science more accessible to females, and to encourage gender equity in these subjects. Despite these efforts, students in the final year of secondary school in most TIMSS nations demonstrated a significant gender gap in the mathematics portion of the general knowledge assessment, with males performing better than females. In the United States, boys' and girls' scores in mathematics general knowledge were not significantly different. The United States was one of three countries (in addition to South Africa and Hungary) among the TIMSS nations which did not have a significant gender gap in mathematics performance (see Table A5.3 in Appendix 5).

Has The Relative International Standing Of The United States In Mathematics At The End Of Secondary School Changed Over Time?

International comparisons over time are difficult. The first international studies of mathematics and science achievement were conducted in the 1960s, and there have been other assessments in each subject since then. However, each assessment has been done differently. A different set of nations participated, different topics in mathematics and science were included in the assessments, the age and type of students sampled in each country changed slightly, and indeed even the borders and names of some of the nations have changed. Furthermore, the field of assessment has matured greatly over the past 30 years, having made many improvements upon the methods of the then-revolutionary early studies. These and other factors complicate comparisons over time and require that any conclusions be necessarily tentative.

In TIMSS, we have seen that U.S. twelfth graders scored below the international average in mathematics general knowledge, and among the lowest of all nations. This international standing was similar to the one reported for U.S. twelfth graders in the IEA First and Second International Mathematics Studies conducted in the 1960s and 1980s. Thus, relative to their international counterparts completing secondary school, it is unlikely that U.S. twelfth graders' standing has changed significantly in mathematics achievement over the past 30 years.

SCIENCE GENERAL KNOWLEDGE ASSESSMENT

The science portion of the general knowledge assessment consisted of about 60 percent multiple choice items and 40 percent free-response items. Items were chosen based on their likelihood of arising in real-life situations and not on their connection to a particular curriculum. Looked at in terms of common science curriculum topics, however, the items covered the topics of earth science, life science, and physical science. On average, for the countries participating in the TIMSS assessment of science general knowledge, these topics are typically covered in about the ninth grade.

How Well Do U.S. Twelfth Graders Do On The Science General Knowledge Assessment?

On the science portion of the general knowledge assessment, U.S. students scored below the international average, and among the lowest scoring of the 21 countries. Figure 5 shows how U.S. students performed on the science general knowledge assessment.

On the assessment of science general knowledge, students at the end of secondary school in 11 countries (Sweden, the Netherlands, Iceland, Norway, Canada, New Zealand, Australia, Switzerland, Austria, Slovenia, and Denmark) outperformed U.S. twelfth graders. Students in 7 countries performed not significantly different from those in the United States (Germany, France, the Czech Republic, the Russian Federation, Italy, Hungary, and Lithuania). Students in Cyprus and South Africa performed below students in the United States.

What Were Students Asked To Do On The Science General Knowledge Assessment?

Three examples of TIMSS science general knowledge assessment items are presented. Table A3.1 in Appendix 3 shows the percentage of students in every participating country responding correctly to each of these example items.

The item shown in Figure 6 requires students to apply scientific principles to develop explanations. Forty-two percent of U.S. students responded correctly to this item. The international average was 61 percent correct. Some students' incorrect responses, such as "because they are sharper and poke into the floor," attributed the damage to sharpness rather than the effects of pressure placed on a small surface area.

The item shown in Figure 7 requires an understanding of causes of pollution. Seventy-eight percent of U.S. twelfth graders responded correctly on this item. The international average was 77 percent.

The item shown in Figure 8 requires knowledge of complex information about the interdependence of life. The U.S. average was 40 percent correct, and the international average was 37 percent. Some students who responded incorrectly to this item were not sufficiently explicit about how a species can regulate the population of its prey.

How Does U.S. Twelfth-Grade Students' Relative Performance In Science Compare To That Of U.S. Eighth-Grade Students?

As in mathematics, it is possible to compare the science performance of all U.S. students at both eighth grade and in the final year of secondary school to the group of 20 countries that participated in both of these portions of TIMSS. Table A5.4 in Appendix 5 displays the standing for each country relative to the international average for science achievement for the two grade levels based on the 20 countries and whether that relative standing was different at the two grade levels. (See Table A5.5 in Appendix 5 for a similar comparison for the 12 countries that participated in TIMSS both at fourth grade and in the science general knowledge assessment at the end of secondary school.)

In science, the United States is one of 7 countries where the standing relative to the international average was lower at the end of secondary schooling than it was at eighth grade. The others were former Communist Bloc countries plus Australia and Germany. Eight countries had a similar standing relative to the international average at both grade levels and 5 had a higher relative standing in the final year of secondary schooling than in eighth grade.

Is There A Gender Gap In Science General Knowledge At The Twelfth Grade?

In the United States, there was a gender gap on the science portion of the twelfth-grade general knowledge assessment. Excluding South Africa, in all other TIMSS nations, including the United States, males performed significantly better than females in science. However, among those countries, the U.S. gender gap in science was one of the smallest (see Table A5.6 in Appendix 5).

Has The Relative International Standing Of The United States In Science At The End Of Secondary School Changed Over Time?

In TIMSS, we have seen that U.S. twelfth graders scored below the international average in science, and among the lowest of all nations. This is basically the same relative international standing reported for U.S. twelfth graders in the IEA First and Second International Science Studies in the 1960s and 1980s. Thus, relative to their international counterparts in the final year of secondary school, it is unlikely that U.S. twelfth graders' standing has changed significantly in science.

How Does The Performance Of U.S. Twelfth Graders In Science Compare To Their Performance In Mathematics?

Although U.S. students scored below the international average and among the lowest of TIMSS nations on both portions of the general knowledge assessments, the U.S. international standing on the general knowledge assessments was slightly higher in science than it was in mathematics. Fourteen countries were significantly higher than the United States in the mathematics general knowledge assessment, while 11 countries outperformed the United States in science general knowledge. This pattern is similar to the fourth- and eighth-grade TIMSS results, in which the U.S. relative international standing was higher in science than it was in mathematics.

Among the major trading partners of the United States that participated in TIMSS at the end of secondary school, students in Canada outperformed the U.S. in both mathematics and science general knowledge. Students in France and Germany outperformed U.S. students in mathematics general knowledge, and performed similar to U.S. students in science.

We have now examined what TIMSS tells us about all students in their final year of secondary school. Next, we turn to an examination of how the advanced students in the United States who were taking or had taken advanced courses in mathematics and science compared to their counterparts in other TIMSS nations.