The Third International Mathematics and Science Study (TIMSS) is the largest and most comprehensive comparative international study of education that has ever been undertaken. TIMSS in the United States was coordinated by the National Center for Education Statistics (NCES) and the National Science Foundation (NSF). The study assessed a half-million students from 41 countries in 30 languages to compare their mathematics and science achievement. This report focuses on the 23 countries that participated in the TIMSS study of students at the end of secondary education.

TIMSS comes at a time when mathematics and science achievement has been designated as an educational priority. One of our eight current National Education Goals is that "by the year 2000, the United States will be first in the world in mathematics and science achievement." In addition, mathematics and science experts have issued major calls for reform in the teaching of their subjects. The National Council of Teachers of Mathematics published Curriculum and Evaluation Standards for School Mathematics¹ in 1989, and Professional Standards for Teaching Mathematics² in 1991. In 1993, the American Association for the Advancement of Science followed suit with Benchmarks for Science Literacy,³ and in 1996, the National Academy of Sciences published National Science Education Standards.⁴

This is the last of three reports in the Pursuing Excellence series. The first report presented initial findings on the eighth grade and was released in November, 1996. The second report presented findings on the fourth grade and was released in June, 1997. This report presents initial findings about the international standing of the United States' twelfth graders relative to students completing secondary school in other countries. It is based on the comparative data published in the report, Mathematics and Science Achievement in the Final Year of Secondary School: IEA's Third International Mathematics and Science Study.⁵ The TIMSS International Study Center at Boston College will release complete data files for the study later this year, which will allow a more extensive examination of student performance in mathematics and science in the participating countries.

WHAT IS TIMSS?

TIMSS is the third comparison of mathematics achievement and third comparison of science achievement carried out by the International Association for the Evaluation of Educational Achievement (IEA). Previous IEA studies of mathematics and science were conducted for each subject separately at various times during the 1960s, 1970s, and 1980s. TIMSS is the first IEA study that has assessed both mathematics and science at the same time. Comparative studies of other subjects, including reading literacy (1992)⁶ and computers in education (1993),⁷ have also been published by the IEA.

TIMSS was designed to focus on students at three different stages of schooling: midway through elementary school, midway through lower secondary school, and at the end of upper secondary school. Initial findings for the 41 countries in the lower secondary school component⁸ and for the 26 countries that participated in the elementary school component⁹ are reported in earlier volumes of the Pursuing Excellence series. This report presents initial findings for the 23 countries in the remaining component of TIMSS, students at the end of secondary education. Findings are presented for four broad areas of performance:

• Mathematics general knowledge^A for all students in the final year of secondary education, including those who had taken advanced courses as well as those who had not;
• Science general knowledge^A for all students in the final year of secondary education, including those who had taken advanced science courses as well as those who had not;
• Advanced mathematics for students in the final year of secondary education who had taken or were taking advanced courses in mathematics; and
• Physics for students in the final year of secondary education who had taken or were taking physics.

For the assessments of mathematics general knowledge and science general knowledge, this report presents results for 21 countries: Australia, Austria, Canada, Cyprus, the Czech Republic, Denmark, France, Germany, Hungary, Iceland, Italy, Lithuania, the Netherlands, New Zealand, Norway, the Russian Federation, Slovenia, South Africa, Sweden, Switzerland, and the United States.

For the assessment of advanced mathematics, results are reported for 16 countries: Australia, Austria, Canada, Cyprus, the Czech Republic, Denmark, France, Germany, Greece, Italy, Lithuania, the Russian Federation, Slovenia, Sweden, Switzerland, and the United States.

For the physics assessment, results are reported for 16 countries: Australia, Austria, Canada, Cyprus, the Czech Republic, Denmark, France, Germany, Greece, Latvia, Norway, the Russian Federation, Slovenia, Sweden, Switzerland, and the United States.

The elementary and middle school components of TIMSS defined eligible students primarily on the basis of age. The elementary school group included students enrolled in the pair of adjacent grades that contained the most 9-year-olds, grades 3 and 4 in the United States and most other countries. The middle school students were in the pair of grades that contained the most 13-year-olds, grades 7 and 8 in the United States and most other countries.

A major goal of the end of secondary school component of TIMSS was to measure what students know by the time they leave the secondary school system. Because countries have different structures for secondary education, the final grade of secondary education in the countries participating in TIMSS may be as low as 9 and as high as 14, depending on the country and program in which the student is enrolled. For the United States, the final year of secondary education is grade 12, and twelfth-grade students were selected for the study. All twelfth graders were eligible for the mathematics and science general knowledge portion of the study. Advanced mathematics students in the United States were defined as twelfth graders who had taken or were taking a full year of a high school course that included the word "calculus" in the title, including pre-calculus. Physics students were twelfth graders who had taken or were taking at least one full year of high school physics. Appendix 1 provides information about how other countries identified students to participate in the study.

Students in both public and private schools were administered the mathematics and science general knowledge assessments, which together were about 1.5 hours in length, and included both multiple-choice and free-response items. In each country, the items were translated into the primary languages of instruction. In the United States, all assessments were administered in English. Testing occurred 2 to 3 months before the end of the 1994-95 school year. Students with special needs and disabilities that would make it difficult for them to take the assessments were excused. Students were allowed to use calculators for all assessments.

Like the other components of TIMSS, participating countries collected data beyond the student assessments. Students completed questionnaires about their experiences in and out of school. School administrators completed questionnaires about school policies and practices. An exploratory curriculum analysis compared mathematics and science curriculum guides and textbooks. It studied subject-matter content, sequencing of topics, and expectations for student performance. Teacher questionnaires were not administered, as some of the graduating students who participated in the study were no longer enrolled in mathematics and science.

TIMSS is the most fair and accurate international comparison of students that has ever been undertaken. In each nation, the students who participated in TIMSS were to be randomly selected to represent all students meeting the grade level or age criteria for each of the three populations. An international curricu-lum analysis was carried out prior to the development of the assessments to ensure that the items would reflect the mathematics and science curricula in the TIMSS countries. To further ensure that the assessments measured knowledge that the world community considers important for students to know, the items were developed by international committees. International monitors carefully checked the translations and visited many classrooms while the assessments were being administered in each of the participating countries to make sure that the instructions were properly followed.

The quality standards for the sampling process in TIMSS were higher than in any previous international comparison of education systems. Maintaining these high standards provided challenges for most of the countries that participated in this portion of TIMSS. Most of the 23 countries - including the United States - experienced difficulties of various types. This is consistent with experience in the United States in conducting assessments at the end of high school. Areas of difficulty included minimizing the extent to which students were excluded from the population eligible for the sample and gaining participation of schools and students after they were selected for the sample.

While most countries had difficulties meeting the sampling standards in this portion of TIMSS, the nature of these difficulties, and the students and schools excluded, are generally well understood. Appendix 1 contains a summary of the TIMSS study guidelines and provides information about sampling and adherence to sampling and other standards in all the countries. All countries in which difficulties arose are shown in parentheses in the figures and tables in this report. The United States is in parentheses because its combined school and student participation rate was 64 percent, below the standard of 75 percent. It is most likely that as a group, schools and students who were selected for TIMSS but did not participate in the assessments in the United States would have had below average scores, thus lowering the U.S. average. This was probably the case as well in other countries having similar difficulties.

Full documentation of the data collection methodologies and statistical analyses used in all the participating countries is available in technical and quality control reports published by the TIMSS International Study Center at Boston College.¹⁰ A list of additional TIMSS reports published to date is contained in Appendix 7.

COMPARING THE UNITED STATES TO OTHER COUNTRIES

Some have argued that comparisons of the performance of U.S. students with students in other countries are inappropriate. One argument is that, in the United States, larger portions of a given age cohort are enrolled in the education system - particularly at the secondary level - than in other countries, resulting in a comparison between our general population and more select groups in other countries. Another argument is that in international comparisons, while the United States tests a sample representative of our general student population, some countries test only those students in elite, college preparatory schools or courses of study. Although these arguments may have been valid in previous studies, neither holds true in TIMSS.

As is discussed in more detail in Chapter 4, the most recent data indicate that in most countries participating in TIMSS, secondary school enrollment rates are similar to that of the United States. Not only do the TIMSS countries have most of their secondary school-age population enrolled in school, the strict quality controls discussed earlier ensured that the sample of students taking the mathematics and science general knowledge assessments were representative of the entire population at the end of secondary school. Thus, for example, in most countries with distinct education "streams," such as academic and vocational, students in all programs were represented in the TIMSS sample. This represents an improvement over previous studies of secondary school achievement, in which some countries only assessed students in certain types of schools or programs.

Of course, there are still many other differences between the secondary school systems of the countries participating in TIMSS. However, since a major goal of this component of TIMSS was to assess how well people entering adulthood understand the mathematics and science needed to function effectively in society, comparing students at the end of secondary school is entirely appropriate. This is because the end of secondary school represents the culmination of each country's attempts to prepare all young people for living in society. Rather than use differences between systems to argue against comparisons, or, at the other extreme, ignore such differences, their relationship to mathematics and science achievement should be explored.

THE TIMSS RESEARCH TEAM

TIMSS was conducted by the IEA, which is a Netherlands-based organization of ministries of education and research institutions from its member countries. The IEA delegated responsibility for overall coordination and management of the TIMSS study to Albert Beaton at the TIMSS International Study Center, located at Boston College. Each of the IEA member nations that made the decision to participate in TIMSS paid for and carried out the data collection in its own country according to the international guidelines. The cost of the international coordination was paid by the National Center for Education Statistics (NCES) of the U.S. Department of Education, the National Science Foundation (NSF), and the Canadian Government.

The United States portion of TIMSS was also funded by NCES and NSF. William Schmidt of Michigan State University was the U.S. National Research Coordinator. Policy decisions on the study were made by the U.S. National Coordinating Committee. NCES monitored the international and U.S. TIMSS projects. The U.S. data collection was carried out by Westat, a private survey research firm. Trevor Williams and Nancy Caldwell were Westat project co-directors. The many advisors to the study are listed in Appendix 6.

The U.S. TIMSS team also included the approximately 10,000 twelfth-grade students who took the assessments, and their principals in 210 schools nationwide. Their cooperation has made this report possible.

WHAT QUESTIONS DOES THIS REPORT ANSWER?

This report answers three basic questions:

• How does the mathematics and science general knowledge of U.S. twelfth graders compare to that of students completing secondary school in other nations?
• How do U.S. high school seniors with instruction in physics and advanced mathematics perform in these subjects in comparison to advanced science and mathematics students in other nations?
• What factors might contribute to the performance of the United States relative to other countries in mathematics and science at the end of secondary school?

Chapter 2 answers the first question. This question is important because it measures what our students know at the end of secondary school compared to similar students in other nations. The findings in this chapter reveal how well our students have been prepared by 12 years of formal schooling for their future as adults in a world that increasingly relies on mathematics, science, and technology.

Chapter 3 answers the second question. Advanced students had taken or were taking higher level mathematics and science courses in secondary school, such as calculus and physics. Many are likely to become our nation's next generation of professionals in fields related to mathematics and science.

Chapter 4 answers the third question. It examines a variety of factors related to schooling and students' lives to see if any of them provide insight into why U.S. students perform as they do relative to students in other countries at the end of secondary school.

1. National Council of Teachers of Mathematics. (1989). Curriculum and Evaluation Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics.

2. National Council of Teachers of Mathematics. (1991). Professional Standards for Teaching Mathematics. Reston, VA: National Council of Teachers of Mathematics.

3. American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy. New York: Oxford Press.

4. National Academy of Sciences. (1996). National Science Education Standards. Washington, DC: National Academy Press.

5. Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1998). Mathematics and Science Achievement in the Final Year of Secondary School. Chestnut Hill, MA: Boston College.

6. Elley, W.B. (1992). How in the World Do Students Read? The Hague, Netherlands: International Association for the Evaluation of Educational Achievement.

7. Pelgrum, H. and Plomp, T. (1993). International IEA Computers in Education Study. New York: Pergamon Press.

8. U.S. Department of Education, National Center for Education Statistics. (1996). Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching, Learning, Curriculum, and Achievement in International Context. NCES 97-198. Washington, DC: .

9. U.S. Department of Education, National Center for Education Statistics. (1997). Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics and Science Achievement in International Context. NCES 97-255. Washington, DC: .

10. Martin, M. and Kelly, D. (1996). Third International Mathematics and Science Study: Technical Report, Volume 1: Design and Development. Chestnut Hill, MA: Boston College; and Martin, M. and Mullis, I.V.S. (1996). Third International Mathematics and Science Study: Quality Assurance in Data Collection. Chestnut Hill, MA: Boston College