THE TIMSS VIDEOTAPE CLASSROOM STUDY

Methods and Findings from an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States

James W. Stigler, Patrick Gonzales, Takako Kawanaka, Steffen Knoll, and Ana Serrano
With the assistance of Eric Derghazarian, Gundula Huber, Fumiko Ichioka, and Nicole Kersting

A Research and Development Report

U.S. Department of Education

Richard W. Riley
Secretary

Office of Educational Research and Improvement

C. Kent McGuire
Assistant Secretary

National Center for Education Statistics

Pascal D. Forgione, Jr.
Commissioner

Early Childhood, International, and Crosscutting Division

Martin E. Orland
Associate Commissioner

International Activities Program

Eugene Owen
Director

National Center for Education Statistics

The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries.

NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to:

National Center for Education Statistics
Office of Educational Research and Improvement
U.S. Department of Education
1990 K Street NW
Washington, DC 20006

e-mail: timss@ed.gov

Suggested Citation:

U.S. Department of Education. National Center for Education Statistics,

The TIMSS Videotape Classroom Study: Methods and Findings from an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States, NCES 1999-074, by James W. Stigler, Patrick Gonzales, Takako Kawanaka, Steffen Knoll, and Ana Serrano.

Washington, DC: U.S. Government Printing Office, 1999

February 1999

Available for downloading at http://nces.ed.gov/timss

Download the Acrobat PDF version of this report:

For high quality printouts please print from the PDF file.

The above report needs to be viewed through Adobe Acrobat Reader

Foreword

The Research and Development (R&D) series of the reports has been initiated:

• To share studies and research that are developmental in nature. The results of such studies may be revised as the work continues and additional data become available.

• To share results of studies that are, to some extent, on the "cutting-edge" of methodological developments. Emerging analytical approaches and new computer software development often permits new, and sometimes controversial, analysis to be done. By participating in "frontier research," we hope to contribute to the resolution of issues and improved analysis.

• To participate in discussions of emerging issues of interest to educational researchers, statisticians, and the Federal statistical community in general. Such reports may document workshops and symposiums sponsored by NCES that address methodological and analytical issues or may share and discuss issues regarding NCES practice, procedures, and standards.

The common theme in all three goals is that these reports present results or discussions that do not reach definitive conclusions at this point in time, either because the data are tentative, the methodology is new and developing, or the topic is one on which there are divergent views. Therefore the techniques and inferences made from the data are tentative and are subject to revision. To facilitate the process of closure on the issues, we invite comment, criticism, and alternatives to what we have done. Such responses should be directed to:

Marilyn M. McMillen
Chief Statistician Statistical Standards Program
National Center for Education Statistics
1900 K Street NW, Suite 9000
Washington DC 20006

Executive Summary

This report presents the methods and preliminary findings of the Videotape Classroom Study, a video survey of eighth-grade mathematics lessons in Germany, Japan, and the United States. This exploratory research project is part of the Third International Mathematics and Science Study (TIMSS). It is the first to collect videotaped records of classroom instruction--in any subject--from national probability samples.

OBJECTIVES

The Videotape Classroom Study had four goals:

1. Provide a rich source of information regarding what goes on inside eighth-grade mathematics classes in the three countries.
2. Develop objective observational measures of classroom instruction to serve as valid quantitative indicators, at a national level, of teaching practices in the three countries.
3. Compare actual mathematics teaching methods in the United States and the other countries with those recommended in current reform documents and with teachers' perceptions of those recommendations.
4. Assess the feasibility of applying videotape methodology in future wider-scale national and international surveys of classroom instructional practices.

SCOPE AND METHODS

The study sample included 231 eighth-grade mathematics classrooms: 100 in Germany, 50 in Japan, and 81 in the United States. The three samples were selected from among the schools and classrooms participating in the 1994-95 TIMSS assessments. They were designed as a nationally representative sample of eighth-grade students in the three countries, although, as will be explained later, some minor deviations arose.

One lesson was videotaped in each classroom at some point during the school year. The specific date for videotaping was determined in consultation with the school and the teacher in order to minimize conflicts with special events such as field trips or school holidays, and to minimize the videographers' travel expenses. Tapes were encoded and stored digitally on CD-ROM and were accessed and analyzed using multimedia database software developed especially for this project. All lessons were transcribed and then analyzed on a number of dimensions by teams of coders who were native speakers of the three languages. Analyses presented here are based on weighted data. The analyses focused on the content and organization of the lessons, as well as on the instructional practices used by teachers during the lessons.

FINDINGS

The video data are vast and will continue to provide rich analysis opportunities for researchers. The findings reported here, while preliminary, reveal a number of differences in instructional practices across the three cultures. These differences fall into four broad categories: (1) How lessons are structured and delivered; (2) What kind of mathematics is presented in the lesson; (3) What kind of mathematical thinking students are engaged in during the lesson; and (4) How teachers view reform.

How Lessons are Structured and Delivered

To understand how lessons are structured it is important first to know what teachers intend students to learn from the lessons. Information gathered from teachers in the video study indicate an important cross-cultural difference in lesson goals. Solving problems is the end goal for the U.S. and German teachers: How well students solve problems is the metric by which success is judged. In Japan, problem solving is assumed to play a different role. Understanding mathematics is the overarching goal; problem solving is merely the context in which understanding can best grow.

Following this difference in goals, we can begin to identify cultural differences in the scripts teachers in each country use to generate their lessons. These different scripts are probably based on different assumptions about the role of problem solving in the lesson, about the way students learn from instruction, and about what the proper role of the teacher should be.

Although the analyses are preliminary, there appears to be a clear distinction between the U.S. and German scripts, on one hand, and the Japanese script, on the other. U.S. and German lessons tend to have two phases: an initial acquisition phase and a subsequent application phase. In the acquisition phase, the teacher demonstrates and/or explains how to solve an example problem. The explanation might be purely procedural (as most often happens in the United States) or may include development of concepts (more often the case in Germany). Yet still, the goal in both countries is to teach students a method for solving the example problem(s). In the application phase, students practice solving examples on their own while the teacher helps individual students who are experiencing difficulty.

Japanese lessons appear to follow a different script. Whereas in German and U.S. lessons instruction comes first, followed by application, in Japanese lessons the order of activity is generally reversed. Problem solving comes first, followed by a time in which students reflect on the problem, share the solution methods they have generated, and jointly work to develop explicit understandings of the underlying mathematical concepts. Whereas students in the U.S. and German classrooms must follow the teacher as he or she leads them through the solution of example problems, the Japanese student has a different job: to invent his or her own solutions, then reflect on those solutions in an attempt to increase understanding.

In addition to these differences in goals and scripts, we also find differences in the coherence of lessons in the three countries. The greatest differences are between U.S. lessons and Japanese lessons. U.S. lessons are less coherent than Japanese lessons if coherence is defined by several criteria: U.S. lessons are more frequently interrupted, both from outside the classroom and within; U.S. lessons contain more topics--within the same lesson--than Japanese lessons; Japanese teachers are more likely to provide explicit links or connections between different parts of the same lesson.

What Kind of Mathematics is Presented

Looking beyond the flow of the lessons, we also find cross-cultural differences in the kind of mathematical content that is presented in the lessons. When viewed in comparison to the content of lessons in the 41 TIMSS countries, the average eighth-grade U.S. lesson in the video sample deals with mathematics at the seventh-grade level by international standards, whereas in Japan the average level is ninth-grade. The content of German lessons averages at the eighth-grade level.

The quality of the content also differs across countries. For example, most mathematics lessons include some mixture of concepts and applications of those concepts to solving problems. How concepts are presented, however, varies a great deal across countries. Concepts might simply be stated, as in "the Pythagorean theorem states that a² + b² = c²," or they might be developed and derived over the course of the lesson. More than three-fourths of German and Japanese teachers develop concepts when they include them in their lessons, compared with about one-fifth of U.S. teachers. None of the U.S. lessons include proofs, whereas 10 percent of German lessons and 53 percent of Japanese lessons include proofs.

Finally, as part of the video study, an independent group of U.S. college mathematics teachers evaluated the quality of mathematical content in a sample of the video lessons. They based their judgments on a detailed written description of the content that was altered for each lesson to disguise the country of origin (deleting, for example, references to currency). They completed a number of in-depth analyses, the simplest of which involved making global judgments of the quality of each lesson's content on a three-point scale (Low, Medium, High). (Quality was judged according to several criteria, including the coherence of the mathematical concepts across different parts of the lesson, and the degree to which deductive reasoning was included.) Whereas 39 percent of the Japanese lessons and 28 percent of the German ones received the highest rating, none of the U.S. lessons received the highest rating. Eighty-nine percent of U.S. lessons received the lowest rating, compared with 11 percent of Japanese lessons.

The Kind of Mathematical Thinking in Which Students are Engaged

When we examine the kind of work students engage in during the lesson we find a strong resemblance between Germany and the United States, with Japan looking distinctly different. Three types of work were coded in the video study: Practicing Routine Procedures, Applying Concepts to Novel Situations, and Inventing New Solution Methods/Thinking. Ninety-six percent of student working time in Germany and 90 percent in the United States is spent in practicing routine procedures, compared with 41 percent in Japan. Japanese students spend 44 percent of their time inventing new solutions that require conceptual thinking about mathematics.

Teachers and Reform

A great deal of effort has been put into the reform of mathematics teaching in the United States in recent years. Numerous documents--examples include the National Council of Teachers of Mathematics Curriculum and Evaluation Standards for School Mathematics (1989) and the National Council of Teachers of Mathematics Professional Standards for Teaching Mathematics (1991)--encourage teachers to change the way they teach, and there is great agreement, at least among mathematics educators, as to what desirable instruction should look like. Although most of the current ideas stated in such documents are not operationalized to the extent that they could be directly coded, it is possible to view some of the indicators developed in the video study in relation to these current ideas.

When the video data are viewed in this way, Japanese teachers, in certain respects, come closer to implementing the spirit of current ideas advanced by U.S. reformers than do U.S. teachers. For example, Japanese lessons include high-level mathematics, a clear focus on thinking and problem solving, and an emphasis on students deriving alternative solution methods and explaining their thinking. In other respects, though, Japanese lessons do not follow such reform guidelines. They include more lecturing and demonstration than even the more traditional U.S. lessons, and we never observed calculators being used in a Japanese classroom.

Regardless of whether Japanese classrooms share features of "reform" classrooms or not, it is quite clear that the typical U.S. classroom does not. Furthermore, the U.S. teachers, when asked if they were aware of current ideas about the best ways to teach mathematics, responded overwhelmingly in the affirmative. Seventy percent of the teachers claim to be implementing such ideas in the very lesson that we videotaped. When asked to justify these claims, the U.S. teachers refer most often to surface features, such as the use of manipulatives or cooperative groups, rather than to the key point of the reform recommendations, which is to focus lessons on high-level mathematical thought. Although some teachers appear to have changed these surface-level characteristics of their teaching, the data collected for this study suggest that these changes have not affected the deeper cultural scripts from which teachers work.

Key Points

Bearing in mind the preliminary nature of these findings, as well as the interpretations of the findings, we can, nevertheless, identify four key points:

1. The content of U.S. mathematics classes requires less high-level thought than classes in Germany and Japan.
2. U.S. mathematics teachers' typical goal is to teach students how to do something, while Japanese teachers' goal is to help them understand mathematical concepts.
3. Japanese classes share many features called for by U.S. mathematics reforms, while U.S. classes are less likely to exhibit these features.
4. Although most U.S. math teachers report familiarity with reform recommendations, relatively few apply the key points in their classrooms.

These initial findings suggest a need for continued analysis of these data on eighth-grade mathematics practices. Caution should be exercised in generalizing to other subjects or grade levels.

Acknowledgments

A project as large as this one would not have been possible without the help of many people. A list, hopefully almost complete, of those who contributed to the project is included as appendix C. Aside from this, there are a number of people who deserve special mention. First, we would like to acknowledge our collaborators: Jürgen Baumert (Max Planck Institute on Education and Human Development, Berlin) and Rainer Lehmann (University of Hamburg) in Germany, and Toshio Sawada at the National Institute of Educational Research in Tokyo, Japan. We also wish to thank Clea Fernandez for her input during the early stages of the project; Michael and Johanna Neubrand for help in understanding German teaching practices; Alfred Manaster, Phillip Emig, Wallace Etterbeek, and Barbara Wells for their work on the mathematics content analyses; and Nancy Caldwell of Westat, who helped out in ways too numerous to mention. Shep Roey cheerfully ran many analyses many times, Lou Rizzo carefully developed the weights applied to the data, and Dave Kastberg supervised the final layout of the report (all of Westat); we greatly appreciate their help. The final report was greatly improved by the hard work of Ellen Bradburn and Christine Welch at the Education Statistics Services Institute. Finally, we would like to acknowledge the major contributions of Lois Peak (U.S. Department of Education), without whom this project would never have been done, and James Hiebert, who has improved the project at every step of the way, from coding and analysis to the writing of this report.