• The Third International Mathematics and Science Study (TIMSS) is the largest, most comprehensive, and most rigorous international comparison of education ever undertaken.

• TIMSS' rich information allows us not only to compare achievement, but also to understand how life in U.S. schools differs from that in other nations.

• This report on eighth-grade students is the first of a series of reports that will present findings on student achievement at the fourth grade, at the end of high school, as well as on various other topics.

OVERVIEW

The Third International Mathematics and Science Study is the largest and most comprehensive comparative international study of education that has ever been undertaken. A half-million students from 41 countries were tested in 30 different languages at five different grade levels to compare their mathematics and science achievement. Intensive studies of students, teachers, schools, curriculum, instruction, and policy issues were also carried out to understand the educational context in which learning takes place.

TIMSS is an important study for those interested in U.S. education. In 1983, the National Commission on Excellence in Education pointed to our nation's low performance in international studies as evidence that we were A Nation at Risk. In 1989, President Bush and the governors of all 50 states adopted the National Goals for Education, one of which was that "by the year 2000, the U.S. will be first in the world in mathematics and science achievement." Mathematics and science experts have issued major calls for reform in the teaching of their subjects. The National Council of Teachers of Mathematics published Curriculum and Evaluation Standards in 1989, and Professional Standards for Teaching Mathematics in 1991. In 1993 the American Association for the Advancement of Science followed suit with Benchmarks for Science Literacy, and in 1996, the National Academy of Sciences published National Science Education Standards.

TIMSS helps us measure progress toward our national goal of improving our children's academic performance in mathematics and science. But TIMSS is much more than a scorecard for the math and science events in the "educational Olympics." It is a diagnostic tool to help us examine our nation's progress toward improvement of mathematics and science education. It was designed to look behind the scorecard to illuminate how our education policies and practices compare to those of the world community.

TIMSS helps us answer the following questions about our nation's mathematics and science learning:

• Are U.S. curricula and expectations for student learning as demanding as those of other nations?

• Is the level of classroom instruction in the U.S. as high as that in other countries?

• Do U.S. teachers receive as much support in their efforts to teach students as their colleagues in other nations?

• Are U.S. students as focused on their studies as their international counterparts?

This report draws from the many reports and parts of the TIMSS study to summarize the initial findings concerning achievement and schooling in the eighth grade. It is part of the first of three waves of TIMSS reports. It will be followed in the next year by a series of reports focusing on the fourth grade, then by a series focusing on the last year of high school. Additional reports on selected topics will be published over the next several years. Much more will be learned as further analysis of the eighth grade data is carried out and findings from grades four and twelve are added.

TIMSS is a fair and accurate comparison of mathematics and science achievement in the participating nations. It is not a comparison of "all of our students, with other nations' best," a charge which some critics have leveled at previous international comparisons. The students who participated in TIMSS were randomly selected to represent all students in their respective nations. The entire assessment process was scrutinized by international technical review committees to ensure its adherence to established standards. Those nations in which irregularities arose are clearly noted in this and other TIMSS reports.

At each step of its development, TIMSS used careful quality control procedures. An international curriculum analysis was carried out prior to the development of the assessments to ensure that the tests reflect the math and science curricula of the variety of TIMSS countries and do not over-emphasize what is taught in only a few. International monitors carefully checked the test translations and visited many classrooms while the tests were being administered in each of the 41 countries to make sure that the instructions were properly followed. The raw data from each country were scrutinized to be sure that no anomalies existed, and all analyses were double checked. Finally, this report has been written and carefully reviewed to avoid over-generalization and inaccuracy.


STUDY DESIGN

TIMSS is the third comparison of mathematics and science achievement carried out by the International Association for the Evaluation of Educational Achievement (IEA). Previous IEA studies of mathematics and science were conducted for each subject separately at various times during the 1960s, 1970s, and 1980s. This is the first time that IEA has assessed both mathematics and science in the same study. Comparative studies of other subjects, including reading literacy (1992)1, and computers in education (1993)2 have also been published by the IEA.

TIMSS was designed to focus on students at three different stages of schooling: midway through elementary school, midway through lower secondary school, and at the end of upper secondary school. Because countries around the world set different ages at which children should begin school, decisions about which students should be tested needed to take both age and grade level into account. The populations tested are listed below. Participation in Population 2 was required of all TIMSS nations, but participation in Populations 1 and 3 was optional.

• Population 1 - those students enrolled in the pair of adjacent grades that contained the most nine-year-olds. (Grades 3 and 4 in the U.S. and most of the world. Grades 2 and 3 in a few nations.)

• Population 2 - those students in the pair of adjacent grades that contained the most thirteen-year-olds at the time of testing. (Grades 7 and 8 in the U.S. and most of the world. Grades 6 and 7 in a few nations.)

• Population 3 - students in their final year of secondary school, whatever their age. (Grade 12 in the U.S. and most nations. Grades 9-13 in some nations.)

In all countries, students in both public and private schools received the TIMSS test. In all but a few of the 41 TIMSS countries, virtually all population 1 and 2 children are enrolled in school and were therefore eligible to take the test. Testing occurred 2 to 3 months before the end of the 1995-96 school year. Students with special needs and disabilities which would make it difficult for them to take the test were excused from the assessment. In each country, the test was translated into the primary language or languages of instruction. All testing in the U.S. was done in the English language.

TIMSS includes five different parts: assessments, questionnaires, curriculum analyses, videotapes of classroom instruction, and case studies of policy topics. The study was designed to bring a variety of different and complementary research methods to bear on important policy questions. The use of multiple methodologies has three major benefits. First, it strengthens the conclusions of the study because researchers are able to cross-check key findings by comparing results based on different research methods. Second, it provides broader information because more different types data are gathered than can be acquired through a single method or instrument. Third, the use of multiple methodologies enriches understanding of the contextual meaning of key findings. Each of the five parts on its own represents an important advance in its field. Taken together, they provide an unprecedented opportunity to understand U.S. mathematics and science education from a new and richer perspective.

At population 2, all 41 TIMSS countries participated in the following three IEA-sponsored parts of the study:

• Math and science assessments - One and a half hours in length, the assessments included both multiple-choice and free-response items. A smaller number of students also completed "hands-on" performance assessments, to be reported later.

• School, teacher, and student questionnaires - Students answered questions about their mathematics and science studies and beliefs. Teachers answered questions on their beliefs about math and science and on teaching practices. School administrators answered questions about school policies and practices.

• Curriculum analysis - This exploratory study compared mathematics and science curriculum guides and textbooks. It studied subject-matter content, sequencing of topics, and expectations for student performance.

In conjunction with these three activities, the United States sponsored two additional parts of TIMSS, which were carried out in Germany, Japan, and the U.S. These three countries are all economic superpowers with close economic and political ties. They also are nations whose educators have learned a great deal from each other in the past, and whose school systems are both similar to and different from each other in important ways. The TIMSS researchers in Germany, Japan, and U.S. collaborated in sharing their assessment and questionnaire data, and in carrying out the following two parts of the study:

• Videotapes of mathematics instruction - In the U.S. and Germany, half of the eighth-grade mathematics classrooms that participated in the main TIMSS study were randomly chosen to be filmed. In Japan, an eighth-grade classroom in a random sample of 50 of the TIMSS schools was chosen to be videotaped. In all three countries teachers were filmed teaching a typical lesson, and these tapes were analyzed to compare teaching techniques and the quality of instruction.

• Ethnographic case studies of key policy topics - A team of 12 bilingual researchers each spent three months in Germany, Japan, or the U.S. observing classrooms and interviewing education authorities, principals, teachers, students, and parents. Topics of study were education standards, methods of dealing with individual differences, the lives and working conditions of teachers, and the role of school in adolescents' lives.

More detail on the findings and methodology of each of these parts of TIMSS can be found in the additional reports listed in Appendix 1.


THE TIMSS RESEARCH TEAM

TIMSS was conducted by the IEA, which is a Netherlands-based organization of ministries of education and research institutions in its member countries. The IEA delegated responsibility for overall coordination and management of the TIMSS study to Professor Albert Beaton at the TIMSS International Study Center, located at Boston College. Each of the 41 IEA member-nations that made the decision to participate in TIMSS paid for and carried out the data collection in its own country according to the international guidelines. The costs of the international coordination were paid by the National Center for Education Statistics of the U.S. Department of Education (NCES), the National Science Foundation (NSF), and the Canadian Government.

TIMSS in the United States was also funded by NCES and NSF. Professor William Schmidt of Michigan State University was the U.S. National Research Coordinator. Policy decisions on the study were made by the U.S. National Coordinating Committee, composed of William Schmidt, Larry Suter of NSF, and Jeanne Griffith, Eugene Owen, and Lois Peak of NCES. Lois Peak monitored the international and U.S. TIMSS data collections. The U.S. data collection was carried out by Westat, a private survey research firm. Trevor Williams and Nancy Caldwell were Westat project co-directors. Professor James Stigler at UCLA managed the TIMSS videotape study of mathematics instruction, and Professor Harold Stevenson at the University of Michigan managed the TIMSS ethnographic case studies. The many advisors to the study are listed in Appendix 2.

The U.S. TIMSS team also includes the nearly 4,000 seventh and 7,000 eighth graders who took the assessment, and their principals and teachers in more than 180 schools nationwide. Their cooperation has made this report possible. Third, fourth, and twelfth graders also took different TIMSS tests, and findings from these parts of the study will be reported during the next year.


ORGANIZATION OF THIS REPORT

This report summarizes early findings from the eighth-grade data based on results from all five parts of the TIMSS study. Both seventh and eighth grade students took the TIMSS test, but this initial report focuses on findings for the eighth grade. Future reports based on a more complete and extensive analysis of the data will provide deeper understanding and investigate relationships between the findings from the different parts of the study. Science teacher questionnaire data used in this report are based on preliminary weights which will be further refined in subsequent reports. Extensive documentation of the data collection methodologies and statistical analyses used in this report are available from NCES, and will be published separately.

This report combines the major findings from each of the five parts of the study into a single story about U.S. eighth-grade mathematics and science achievement in comparative perspective. In some respects, results for mathematics and science are similar. The report focuses more on mathematics for two reasons. First, the way in which the subject is taught makes it easier to compare across countries. Second, TIMSS contains more data about mathematics because the videotapes of classroom instruction were conducted only in this subject. Discussion of findings notes where the results in science differ from those in math. This report describes the U.S. against the backdrop of the 41 TIMSS countries, with a special attention to comparisons with Germany and Japan, because we have more information on these countries.

Chapter 1 draws from the results of the student assessments to describe how U.S. students perform in mathematics and science. Succeeding chapters focus on factors which may have an important influence on achievement, and describe how our nation's schools, teachers, and students compare to those in other countries. Chapter 2 examines educational standards and the curriculum, based on data from the curriculum analysis, case studies, videotape study, and questionnaires. Chapter 3 focuses on how teachers actually teach that curriculum, drawing from results of the videotape study and questionnaires. Chapter 4 examines the working life of teachers, based upon findings from the case studies and questionnaires. Chapter 5 describes the lives of students, both in and out of school, based upon case study and questionnaire data. The Conclusions at the end of the report looks across all of the findings for insights about factors associated with student performance and indicates questions for further study.