Chapter 2 has shown us how U.S. twelfth graders performed in mathematics and science general knowledge in comparison to students at the close of their secondary school studies in other countries. It shows us the level of general mathematics and scientific knowledge of the entire sampled student population. But because advances in science and technology are playing a greater role in shaping the future of our nation and our world, it is useful to look beyond the general levels of science and mathematics general knowledge and focus on the advanced levels of knowledge of those who are likely to become our next generation of professionals in fields related to mathematics and science.

Therefore, in addition to the TIMSS assessments of science and mathematics general knowledge, other assessments were created to compare the achievement of students taking advanced science and mathematics courses. Physics was selected by the participating countries for the advanced science assessment because "it is the branch of science most closely associated with mathematics," and because it was viewed as coming "closest to the essential elements of natural science."¹¹ Students were allowed to use calculators on the assessments and relevant formulas were provided.

The numbers of students participating in the physics and advanced mathematics assessments were generally much smaller (one-third to one-half as many) than in the mathematics and science general knowledge assessments. As a result, estimates of a country's average scores in the physics and advanced mathematics assessments have larger margins of error than in the mathematics and science general knowledge assessments. We can say with 95 percent confidence that comparisons of other countries' scores to those of the United States are accurate plus or minus 20 to 40 points on advanced mathematics, and 15 to 65 points on physics, depending on the size and design of the samples in other countries. Comparisons of the United States with the international average are accurate plus or minus about 12 points for advanced mathematics and 7 points for physics. (Table A2.2 in Appendix 2 contains a list of national average scores and standard errors.)

ADVANCED MATHEMATICS ASSESSMENT

An assessment of advanced mathematics was given to a sample of students taking advanced coursework in mathematics. In the United States, these were students who had taken or were taking a full year of a high school course that included the word "calculus" in the title. This included calculus, pre-calculus, Advanced Placement calculus, and calculus and analytic geometry. It should be noted, however, that the advanced mathematics assessment was not primarily a calculus assessment. About one-quarter of the items were in the content area of calculus. Other content areas included in the assessment were: numbers, equations, and functions; validation and structure; probability and statistics; and geometry. Sub-scales were created for the geometry; calculus; and numbers, equations, and functions content areas. The number of items in the other two categories was too small to obtain reliable scores so separate sub-scales were not developed for them. On the advanced mathematics assessment, three-quarters of the items were multiple choice and one-quarter free response.

Fewer countries participated in the advanced mathematics assessment than in the general knowledge assessments. Among the 21 countries which participated in the mathematics and science general knowledge assessments, six countries (Hungary, Iceland, the Netherlands, New Zealand, Norway, and South Africa) did not administer the advanced mathematics assessment. Greece, which did not participate in the mathematics and science general knowledge assessments, participated in the advanced mathematics assessment. As a result, 16 countries participated in the advanced mathematics assessment.

The goal of the advanced mathematics assessment was to compare the mathematics performance of students in the most advanced 10 to 20 percent of their age cohort across nations. Countries were asked to identify these students using definitions appropriate for their own education systems. In the United States, in order to meet the criterion of representing 10 to 20 percent of the age cohort, students whose highest mathematics course was pre-calculus were included along with students who had studied or were studying calculus.

In two countries, the Russian Federation and Lithuania, the advanced mathematics students constituted less than 5 percent of their age cohort; in Austria, Germany, and Slovenia, they constituted more than 20 percent. In the remaining 11 countries - including the United States - students in the advanced mathematics assessment were representative of about 10 to 20 percent of their age cohort. Table A5.7 in Appendix 5 contains the estimated percentages of the age cohort in each country represented by students who took the advanced mathematics assessment.

How Do Our Twelfth Graders With Advanced Mathematics Instruction Compare To Advanced Mathematics Students In Other Countries?

The performance of U.S. twelfth-grade advanced mathematics students was among the lowest of the 16 TIMSS nations who administered the assessment to a comparable population of their advanced mathematics students and below the international average. Figure 9 shows that 11 nations outperformed the United States, while U.S. scores were not significantly different from those of 4 other nations. No countries scored below the United States on the assessment of advanced mathematics.

U.S. advanced mathematics students included those who had completed or were completing pre-calculus, calculus, calculus and analytic geometry, or Advanced Placement calculus, repre-senting about 14 percent of the school-completing age cohort in the United States. If we compared only those U.S. students who had taken or were taking calculus or Advanced Placement calculus against all the advanced mathematics students in other countries, how did our calculus students perform?

How Do U.S. Twelfth Graders With Calculus Or Advanced Placement Calculus Compare To All Advanced Mathematics Students In Other Countries?

U.S. twelfth graders with calculus or Advanced Placement calculus instruction represented about 7 percent of the U.S. age cohort. These students did perform better in the assessment than the larger U.S. group that also included students whose highest course was pre-calculus.

Advanced mathematics students in 6 countries (France, the Russian Federation, Switzerland, Denmark, Cyprus, and Lithuania) outperformed calculus and AP calculus students in the U.S. Figure 10 shows that the performance of U.S. twelfth graders with calculus or Advanced Placement calculus instruction was not significantly different from the international average and 7 of the 16 TIMSS nations that administered the assessment to their advanced mathematics students. Our scores were significantly higher than those of two other nations (Germany and Austria).

The performance of U.S. twelfth graders with Advanced Placement calculus instruction, who represent about 5 percent of the U.S. age cohort was significantly higher than the performance of advanced mathematics students in 5 other countries. Figure 11 shows that one nation (France) outperformed the United States, while our scores were not significantly different from 9 other countries and the international average. Thus, the most advanced mathematics students in the United States, about 5 percent of the total age cohort, performed similarly to 10 to 20 percent of the age cohort in most of the other countries.

How Do U.S. Students Score In The Different Content Areas Of Advanced Mathematics?

Representing student achievement in advanced mathematics as a total score is a useful way to summarize achievement. However, the advanced mathematics assessment contained different content areas, which are emphasized and se-quenced differently in curricula around the world. Based on national priorities, some content areas have been studied more than others in different countries by the time these students are ready to graduate from secondary school.

The TIMSS advanced mathematics assessment included sets of items designed to sample students' ability to do work in the following areas:

Numbers, Equations, and Functions: Complex numbers and their properties; permutations and combinations; equations and formulas; and patterns, relations, and functions.

Calculus:
Infinite processes; and change.

Geometry:
Basic geometry; coordinate geometry; polygons and circles; and three-dimensional geometry.

Figure 12 shows that in all of the content areas of advanced mathematics, U.S. students' performance was among the lowest of the TIMSS nations.

Among the content areas, U.S. students' performance was relatively weakest in geometry: no country scored similar to or below the United States. In numbers, equations, and functions, as well as in calculus, fewer countries (11 countries) scored above the United States. See Table A4.1 in Appendix 4 for U.S. AP and non-AP calculus students' scores by content area.

What Were Students Asked To Do On The Advanced Mathematics Assessment?

There are three examples of advanced mathematics assessment items. Table A3.2 in Appendix 3 shows the percentage of students in every country responding correctly to each of these example items.

An example of a geometry item is shown in Figure 13. This item required students to prove and justify the given statement. The international average for this item was 48 percent. Nineteen percent of U.S. students responded at least partially correctly. Over one-third of U.S. students did not receive credit for this item due to incorrect argumentation and/or including more than one incorrect geometric fact, step, or reason.

Figure 14 is an example of a probability and statistics item. Of U.S. students, 62 percent responded correctly. The international average was 50 percent correct. Some students who responded incorrectly to this item chose "E," perhaps simply counting all of the even numbers between four and twenty-four, rather than correctly counting only the factors of 24 that are divisible by four or six.

The calculus item shown in Figure 15, that required students to demonstrate their understanding of integrals, proved to be a difficult item for most students, including U.S. students. Twenty-seven percent of U.S. students responded correctly to this item, and the international average was about 35 percent. Many students who responded incorrectly apparently did not recognize that if a curve lies above the x-axis, the integral represents the area under the curve, and if the curve lies below the x-axis, the integral represents the negative of the area between the curve and the x-axis.

Is There A Gender Gap In Advanced Mathematics At The Twelfth Grade?

In the United States, twelfth-grade males outperformed twelfth-grade females in advanced mathematics. The United States was one of the 11 TIMSS nations in which a gender gap existed. No significant gender gap existed in the other 5 countries. For the United States and 7 other countries, there was a significant gender gap existing in all 3 advanced mathematics content areas. (See Tables A5.8 and A5.9 in Appendix 5.)

PHYSICS ASSESSMENT

The TIMSS physics assessment included questions about mechanics; electricity and magnetism; particle, quantum and other types of modern physics; heat; and wave phenomena. In the United States, the population that took the assessment was twelfth graders who had taken or were taking at least one year-long course in physics. This included physics I, physics II, advanced physics, and Advanced Placement physics.

Fewer countries participated in the physics assessment than in the general knowledge assessments. Among the 21 countries which participated in the mathematics and science general knowledge assessments, 7 countries (Hungary, Iceland, Italy, Lithuania, the Netherlands, New Zealand, and South Africa) did not administer the physics assessment. Greece and Latvia, which did not participate in the general knowledge assessments, participated in the physics assessment. As a result, 16 countries participated in the physics assessment.

In general, countries identified similar percentages of an age cohort as appropriate for the physics assessment. Although countries used their own definitions to identify advanced science students, for 11 of the 16 countries, including the United States, these students represented about 10 to 20 percent of their age cohort. The only exceptions to this pattern were Denmark, Latvia, and the Russian Federation, where physics students represented less than 5 percent of the age cohort, and Austria and Slovenia, where the students represented more than 20 percent of the age cohort. (Table A5.7 in Appendix 5 contains the percentages of the age cohort in each country represented by students who took the physics assessment.)

How Do U.S. Twelfth Graders With Physics Instruction Compare To Advanced Science Students In Other Countries?

The performance of U.S. twelfth-grade physics students was among the lowest of the 16 TIMSS nations that administered the assessment to a comparable population of their students and below the international average. Figure 16 shows that 14 nations outperformed the United States, while our scores were not significantly different from those of one other nation. No countries scored below the United States on the physics assessment. One interesting aspect of the scores on the physics assessment is that there was less variation in the scores among U.S. students than in 13 of the other 15 countries. The range of U.S. students scores was relatively narrow - 189 points between the 5th and 95th percentile compared to an average difference of 293 points for all 16 countries.

In the United States, the population that took the assessment were twelfth graders who had completed or were completing physics I, physics II, advanced physics, or Advanced Placement physics, representing about 14 percent of the age cohort. If we compared only those U.S. students who had taken or were taking Advanced Placement physics with all the advanced science students in other countries, how did U.S. AP physics students perform?

How Do U.S. Twelfth Graders With Advanced Placement Physics Instruction Compare To Advanced Science Students In Other Countries?

U.S. twelfth graders with Advanced Placement physics represented about 1 percent of the age cohort in the United States. U.S. students who had taken or were taking Advanced Placement physics were outperformed by advanced science students in fewer nations than were the students in our larger group of physics students. Figure 17 shows that U.S. Advanced Placement physics students scored below the international average and advanced science students in 6 nations, while the average U.S. score was significantly higher than that of one other nation. The performance of U.S. twelfth graders with Advanced Placement physics instruction was no different from the performance of 8 of the 16 TIMSS nations that administered the assessment to their advanced science students. However, U.S. Advanced Placement physics students represented a much smaller proportion of the age cohort in the United States than did the advanced science students in most of the other countries.

How Do U.S. Students Score In The Different Content Areas Of Physics?

Representing student achievement in physics as a total score is a useful way to summarize achievement. However, the physics assessment contained different content areas, which are emphasized and sequenced differently in curricula around the world. Based on national priorities, and sequencing of physics instruction for advanced students at the secondary level, some content areas have been studied more than others in various countries by the time these students graduate from secondary school.

The TIMSS physics assessment included sets of items designed to sample students' ability to do work in the following areas:

Mechanics: Dynamics of motion; time, space and motion; types of forces; and fluid behavior.

Electricity/magnetism: Electricity; and magnetism.

Heat: Physical changes; energy types, sources and conversions; heat and temperature; and kinetic theory.

Wave phenomena: Sound and vibration; light; and wave phenomena.

Modern physics: Nuclear chemistry; quantum theory and fundamental particles; astrophysics; subatomic particles; and relativity theory.

Figure 18 shows that in all five physics content areas, U.S. students' performance was among the lowest of the TIMSS nations. In two content areas, one nation scored below the United States. Students in Austria scored below students in the United States in the content area of heat and students in Cyprus scored below U.S. students in the area of modern physics. Among the content areas, U.S. students performed the poorest in the content areas of mechanics and electricity/magnetism, in terms of the number of countries outperforming the United States. See Table A4.2 in Appendix 4 for U.S. AP and non-AP physics students' scores by content area.

What Were Advanced Science Students Asked To Do On The Physics Assessment?

Figures 19, 20 and 21 provide three examples of physics assessment items. Table A3.2 in Appendix 3 shows the percentage of students in every country responding correctly to each of these example items.

Figure 19 is an example of a mechanics item. Forty-one percent of U.S. physics students responded correctly to this item. The international average was about 70 percent. Some students who responded incorrectly appear not to have recognized that the pressure of the water would cause the horizontal placement of the streams to differ. Figure 20 is an example of a heat item. Forty-nine percent of U.S. physics students chose the correct answer. The international average on this item was 41 percent correct. Many students who responded incorrectly chose "A."

A third example of a physics item concerns wave phenomena, as shown in Figure 21. This proved to be a difficult item for most students, including U.S. students. Twelve percent of U.S. physics students received at least partial credit for this item, and the international average was approximately 37 percent correct. Some students who responded incorrectly to this item did not adequately distinguish between the loudness of the sound and a change in the wave frequency.

Is There A Gender Gap In Physics At The Twelfth Grade?

In the United States, as in all the other TIMSS nations except Latvia, twelfth-grade males outperformed twelfth-grade females in physics. In the United States, this gender gap existed in 4 of the 5 content areas of physics included in the TIMSS assessment (all except heat). More than three-quarters of the countries had a significant gender gap in the content areas of mechanics, wave phenomena, and modern physics. (See Tables A5.10 and A5.11 in Appendix 5.)

How Does U.S. Student Performance In Physics Compare To That In Advanced Mathematics?

Unlike our performance on the general knowledge portion of the assessment (where U.S. students' relative performance was stronger in science than it was in mathematics), U.S. performance on the physics assessment was weaker relative to other countries than on the advanced mathematics assessment. Fourteen countries scored above the U.S. in the physics assessment, while fewer countries (11 countries) outperformed the U.S. in the advanced mathematics assessment.

The relationship between performance in physics and advanced mathematics might be more similar to the pattern in general knowledge assessments (with students performing better in science than in mathematics) if TIMSS had assessed other content areas of science such as life science or environmental issues and the nature of science, as was done in eighth grade. Among the content areas of the science assessment given to eighth graders, U.S. students' performance was weakest in physics and strongest in life science and in environmental issues and the nature of science.

11. Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1998). Mathematics and Science Achievement in the Final Year of Secondary School. Chestnut Hill, MA: Boston College.