CHAPTER 1: Introduction
The Third International Mathematics and Science Study (TIMSS)
is the largest and most comprehensive comparative international
study of education that has ever been undertaken. TIMSS in the
United States was coordinated by the National Center for Education
Statistics (NCES) and the National Science Foundation (NSF). The
study assessed a half-million students from 41 countries in 30
languages to compare their mathematics and science achievement.
This report focuses on the 23 countries that participated in the
TIMSS study of students at the end of secondary education.
TIMSS comes at a time when mathematics and science achievement
has been designated as an educational priority. One of our eight
current National Education Goals is that "by the year 2000, the
United States will be first in the world in mathematics and science
achievement." In addition, mathematics and science experts have
issued major calls for reform in the teaching of their subjects.
The National Council of Teachers of Mathematics published Curriculum and Evaluation Standards for School Mathematics1 in 1989, and Professional Standards for Teaching Mathematics2 in 1991. In 1993, the American Association for the Advancement
of Science followed suit with Benchmarks for Science Literacy,3 and in 1996, the National Academy of Sciences published National Science Education Standards.4
This is the last of three reports in the Pursuing Excellence series. The first report presented initial findings on the eighth
grade and was released in November, 1996. The second report presented
findings on the fourth grade and was released in June, 1997. This
report presents initial findings about the international standing
of the United States' twelfth graders relative to students completing
secondary school in other countries. It is based on the comparative
data published in the report, Mathematics and Science Achievement in the Final Year of Secondary
School: IEA's Third International Mathematics and Science Study.5 The TIMSS International Study Center at Boston College will release
complete data files for the study later this year, which will
allow a more extensive examination of student performance in mathematics
and science in the participating countries.
WHAT IS TIMSS?
TIMSS is the third comparison of mathematics achievement and third
comparison of science achievement carried out by the International
Association for the Evaluation of Educational Achievement (IEA).
Previous IEA studies of mathematics and science were conducted
for each subject separately at various times during the 1960s,
1970s, and 1980s. TIMSS is the first IEA study that has assessed
both mathematics and science at the same time. Comparative studies
of other subjects, including reading literacy (1992)6 and computers in education (1993),7 have also been published by the IEA.
TIMSS was designed to focus on students at three different stages
of schooling: midway through elementary school, midway through
lower secondary school, and at the end of upper secondary school.
Initial findings for the 41 countries in the lower secondary school
component8 and for the 26 countries that participated in the elementary
school component9 are reported in earlier volumes of the Pursuing Excellence series. This report presents initial findings for the 23 countries
in the remaining component of TIMSS, students at the end of secondary
education. Findings are presented for four broad areas of performance:
- Mathematics general knowledgeA for all students in the final year of secondary education, including
those who had taken advanced courses as well as those who had
- Science general knowledgeA for all students in the final year of secondary education, including
those who had taken advanced science courses as well as those
who had not;
- Advanced mathematics for students in the final year of secondary
education who had taken or were taking advanced courses in mathematics;
- Physics for students in the final year of secondary education
who had taken or were taking physics.
A The terms "mathematics general knowledge" and "science general
knowledge" used throughout this report are equivalent to the terms
"mathematics literacy" and "science literacy" used in the international
report on achievement in the final year of secondary school published
by Boston College.
For the assessments of mathematics general knowledge and science
general knowledge, this report presents results for 21 countries:
Australia, Austria, Canada, Cyprus, the Czech Republic, Denmark,
France, Germany, Hungary, Iceland, Italy, Lithuania, the Netherlands,
New Zealand, Norway, the Russian Federation, Slovenia, South Africa,
Sweden, Switzerland, and the United States.
For the assessment of advanced mathematics, results are reported
for 16 countries: Australia, Austria, Canada, Cyprus, the Czech
Republic, Denmark, France, Germany, Greece, Italy, Lithuania,
the Russian Federation, Slovenia, Sweden, Switzerland, and the
For the physics assessment, results are reported for 16 countries:
Australia, Austria, Canada, Cyprus, the Czech Republic, Denmark,
France, Germany, Greece, Latvia, Norway, the Russian Federation,
Slovenia, Sweden, Switzerland, and the United States.
The elementary and middle school components of TIMSS defined eligible
students primarily on the basis of age. The elementary school
group included students enrolled in the pair of adjacent grades
that contained the most 9-year-olds, grades 3 and 4 in the United
States and most other countries. The middle school students were
in the pair of grades that contained the most 13-year-olds, grades
7 and 8 in the United States and most other countries.
A major goal of the end of secondary school component of TIMSS
was to measure what students know by the time they leave the secondary
school system. Because countries have different structures for secondary education,
the final grade of secondary education in the countries participating
in TIMSS may be as low as 9 and as high as 14, depending on the
country and program in which the student is enrolled. For the
United States, the final year of secondary education is grade
12, and twelfth-grade students were selected for the study. All
twelfth graders were eligible for the mathematics and science
general knowledge portion of the study. Advanced mathematics students
in the United States were defined as twelfth graders who had taken
or were taking a full year of a high school course that included
the word "calculus" in the title, including pre-calculus. Physics
students were twelfth graders who had taken or were taking at
least one full year of high school physics. Appendix 1 provides
information about how other countries identified students to participate
in the study.
Students in both public and private schools were administered
the mathematics and science general knowledge assessments, which
together were about 1.5 hours in length, and included both multiple-choice
and free-response items. In each country, the items were translated
into the primary languages of instruction. In the United States,
all assessments were administered in English. Testing occurred
2 to 3 months before the end of the 1994-95 school year. Students
with special needs and disabilities that would make it difficult
for them to take the assessments were excused. Students were allowed
to use calculators for all assessments.
Like the other components of TIMSS, participating countries collected
data beyond the student assessments. Students completed questionnaires
about their experiences in and out of school. School administrators
completed questionnaires about school policies and practices.
An exploratory curriculum analysis compared mathematics and science
curriculum guides and textbooks. It studied subject-matter content,
sequencing of topics, and expectations for student performance.
Teacher questionnaires were not administered, as some of the graduating
students who participated in the study were no longer enrolled
in mathematics and science.
TIMSS is the most fair and accurate international comparison of
students that has ever been undertaken. In each nation, the students who participated in TIMSS were to
be randomly selected to represent all students meeting the grade
level or age criteria for each of the three populations. An international
curricu-lum analysis was carried out prior to the development
of the assessments to ensure that the items would reflect the
mathematics and science curricula in the TIMSS countries. To further
ensure that the assessments measured knowledge that the world
community considers important for students to know, the items
were developed by international committees. International monitors
carefully checked the translations and visited many classrooms
while the assessments were being administered in each of the participating
countries to make sure that the instructions were properly followed.
The quality standards for the sampling process in TIMSS were higher
than in any previous international comparison of education systems.
Maintaining these high standards provided challenges for most
of the countries that participated in this portion of TIMSS. Most
of the 23 countries - including the United States - experienced
difficulties of various types. This is consistent with experience
in the United States in conducting assessments at the end of high
school. Areas of difficulty included minimizing the extent to
which students were excluded from the population eligible for
the sample and gaining participation of schools and students after
they were selected for the sample.
While most countries had difficulties meeting the sampling standards
in this portion of TIMSS, the nature of these difficulties, and
the students and schools excluded, are generally well understood.
Appendix 1 contains a summary of the TIMSS study guidelines and provides
information about sampling and adherence to sampling and other
standards in all the countries. All countries in which difficulties
arose are shown in parentheses in the figures and tables in this
report. The United States is in parentheses because its combined
school and student participation rate was 64 percent, below the
standard of 75 percent. It is most likely that as a group, schools
and students who were selected for TIMSS but did not participate
in the assessments in the United States would have had below average
scores, thus lowering the U.S. average. This was probably the
case as well in other countries having similar difficulties.
Full documentation of the data collection methodologies and statistical
analyses used in all the participating countries is available
in technical and quality control reports published by the TIMSS
International Study Center at Boston College.10 A list of additional TIMSS reports published to date is contained
in Appendix 7.
COMPARING THE UNITED STATES TO OTHER COUNTRIES
Some have argued that comparisons of the performance of U.S. students
with students in other countries are inappropriate. One argument
is that, in the United States, larger portions of a given age
cohort are enrolled in the education system - particularly at
the secondary level - than in other countries, resulting in a
comparison between our general population and more select groups
in other countries. Another argument is that in international
comparisons, while the United States tests a sample representative
of our general student population, some countries test only those
students in elite, college preparatory schools or courses of study.
Although these arguments may have been valid in previous studies,
neither holds true in TIMSS.
As is discussed in more detail in Chapter 4, the most recent data
indicate that in most countries participating in TIMSS, secondary
school enrollment rates are similar to that of the United States.
Not only do the TIMSS countries have most of their secondary school-age
population enrolled in school, the strict quality controls discussed
earlier ensured that the sample of students taking the mathematics
and science general knowledge assessments were representative
of the entire population at the end of secondary school. Thus, for example, in most countries with distinct education "streams,"
such as academic and vocational, students in all programs were
represented in the TIMSS sample. This represents an improvement over previous studies of secondary
school achievement, in which some countries only assessed students
in certain types of schools or programs.
Of course, there are still many other differences between the
secondary school systems of the countries participating in TIMSS.
However, since a major goal of this component of TIMSS was to
assess how well people entering adulthood understand the mathematics
and science needed to function effectively in society, comparing
students at the end of secondary school is entirely appropriate.
This is because the end of secondary school represents the culmination
of each country's attempts to prepare all young people for living
in society. Rather than use differences between systems to argue
against comparisons, or, at the other extreme, ignore such differences,
their relationship to mathematics and science achievement should
THE TIMSS RESEARCH TEAM
TIMSS was conducted by the IEA, which is a Netherlands-based organization
of ministries of education and research institutions from its
member countries. The IEA delegated responsibility for overall
coordination and management of the TIMSS study to Albert Beaton
at the TIMSS International Study Center, located at Boston College.
Each of the IEA member nations that made the decision to participate
in TIMSS paid for and carried out the data collection in its own
country according to the international guidelines. The cost of
the international coordination was paid by the National Center
for Education Statistics (NCES) of the U.S. Department of Education,
the National Science Foundation (NSF), and the Canadian Government.
The United States portion of TIMSS was also funded by NCES and
NSF. William Schmidt of Michigan State University was the U.S.
National Research Coordinator. Policy decisions on the study were
made by the U.S. National Coordinating Committee. NCES monitored
the international and U.S. TIMSS projects. The U.S. data collection
was carried out by Westat, a private survey research firm. Trevor
Williams and Nancy Caldwell were Westat project co-directors.
The many advisors to the study are listed in Appendix 6.
The U.S. TIMSS team also included the approximately 10,000 twelfth-grade
students who took the assessments, and their principals in 210
schools nationwide. Their cooperation has made this report possible.
WHAT QUESTIONS DOES THIS REPORT ANSWER?
This report answers three basic questions:
- How does the mathematics and science general knowledge of U.S.
twelfth graders compare to that of students completing secondary
school in other nations?
- How do U.S. high school seniors with instruction in physics and
advanced mathematics perform in these subjects in comparison to
advanced science and mathematics students in other nations?
- What factors might contribute to the performance of the United
States relative to other countries in mathematics and science
at the end of secondary school?
Chapter 2 answers the first question. This question is important because
it measures what our students know at the end of secondary school
compared to similar students in other nations. The findings in
this chapter reveal how well our students have been prepared by
12 years of formal schooling for their future as adults in a world
that increasingly relies on mathematics, science, and technology.
Chapter 3 answers the second question. Advanced students had taken or were
taking higher level mathematics and science courses in secondary
school, such as calculus and physics. Many are likely to become
our nation's next generation of professionals in fields related
to mathematics and science.
Chapter 4 answers the third question. It examines a variety of factors
related to schooling and students' lives to see if any of them
provide insight into why U.S. students perform as they do relative
to students in other countries at the end of secondary school.
WORKS CITED in this chapter
1. National Council of Teachers of Mathematics. (1989). Curriculum and Evaluation Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics.
2. National Council of Teachers of Mathematics. (1991). Professional Standards for Teaching Mathematics. Reston, VA: National Council of Teachers of Mathematics.
3. American Association for the Advancement of Science. (1993).
Benchmarks for Science Literacy. New York: Oxford Press.
4. National Academy of Sciences. (1996). National Science Education Standards. Washington, DC: National Academy Press.
5. Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J.,
Kelly, D.L., and Smith, T.A. (1998). Mathematics and Science Achievement in the Final Year of Secondary
School. Chestnut Hill, MA: Boston College.
6. Elley, W.B. (1992). How in the World Do Students Read? The Hague, Netherlands: International Association for the Evaluation
of Educational Achievement.
7. Pelgrum, H. and Plomp, T. (1993). International IEA Computers in Education Study. New York: Pergamon Press.
8. U.S. Department of Education, National Center for Education
Statistics. (1996). Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics
and Science Teaching, Learning, Curriculum, and Achievement in
International Context. NCES 97-198. Washington, DC: .
9. U.S. Department of Education, National Center for Education
Statistics. (1997). Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics
and Science Achievement in International Context. NCES 97-255. Washington, DC: .
10. Martin, M. and Kelly, D. (1996). Third International Mathematics and Science Study: Technical Report,
Volume 1: Design and Development. Chestnut Hill, MA: Boston College; and Martin, M. and Mullis,
I.V.S. (1996). Third International Mathematics and Science Study: Quality Assurance
in Data Collection. Chestnut Hill, MA: Boston College
Complete list of works cited
Questions, problems or comments with this Web site?