Vol 1, Issue 2, Topic: International Statistics

The TIMSS Videotape Classroom Study: Methods and Findings From an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States

By: James W. Stigler, Patrick Gonzales, Takako Kawanaka, Steffen Knoll, and Ana Serrano

• Important Note
• Objectives
• Scope and Methods
• Findings
• References

Research and Development Reports are intended to

• Share studies and research that are developmental in nature.
• Share results of studies that are on the cutting edge of methodological developments.
• Participate in discussions of emerging issues of interest to researchers.

This report presents the methods and preliminary findings of the Videotape Classroom Study, a video survey of eighth-grade mathematics lessons in Germany, Japan, and the United States. Part of the Third International Mathematics and Science Study (TIMSS), this exploratory research project is the first study to collect videotaped records of classroom instruction—in any subject—from national probability samples.

The Videotape Classroom Study had four goals:

• Provide a rich source of information regarding what goes on inside eighth-grade mathematics classes in the three countries.
• Develop objective observational measures of classroom instruction to serve as valid quantitative indicators, at a national level, of teaching practices in the three countries.
• Compare actual mathematics teaching methods in the United States and the other countries with those recommended

• in current reform documents and with teachers' perceptions of those recommendations.
• Assess the feasibility of applying videotape methodology in future wider scale national and international surveys of classroom instructional practices.

The study sample included 231 eighth-grade mathematics classrooms: 100 in Germany, 50 in Japan, and 81 in the United States. The three samples were selected from among the schools and classrooms participating in the 1994-95 TIMSS assessments. They were designed as nationally representative samples of eighth-grade students in the three countries, although some minor deviations arose. In the United States, the TIMSS sample consisted of 109 schools, each of which was paired with a school that had similar characteristics. Forty of the sampled schools refused to participate. Twelve of these schools were replaced with schools from the "paired" sample. Thus, the final video sample in the United States consisted of 81 schools. The high refusal rate among originally sampled U.S. schools should be kept in mind as a potential source of sampling bias. In the Japanese sample, when there was more than one eighth-grade class in a school, the principal exercised discretion in the choice of classrooms to be videotaped.

One lesson was videotaped in each classroom at some point during the school year. The specific date for videotaping was determined in consultation with the school and the teacher in order to minimize conflicts with special events, such as field trips or school holidays, and to minimize the videographers' travel expenses. Tapes were encoded and stored digitally on CD-ROM and were accessed and analyzed using multimedia database software developed especially for this project. All lessons were transcribed and then analyzed on a number of dimensions by teams of coders who were native speakers of the three languages. Analyses presented here are based on weighted data. The analyses focused on the content and organization of the lessons, as well as on the instructional practices used by teachers during the lessons.

The video data are vast and will continue to provide rich analysis opportunities for researchers. The findings reported here, while preliminary, reveal a number of differences in instructional practices across the three cultures. These differences fall into four broad categories: (1) how lessons are structured and delivered; (2) what kind of mathematics is presented in the lessons; (3) what kind of mathematical thinking students are engaged in during the lessons; and (4) how teachers view reform.

To understand how lessons are structured, it is important first to know what teachers intend students to learn from the lessons. Information gathered from teachers in the video study indicates an important cross-cultural difference in lesson goals. Solving problems is the end goal for the U.S. and German teachers: how well students solve problems is the metric by which success is judged. In Japan, problem solving is assumed to play a different role. Understanding mathematics is the overarching goal; problem solving is merely the context in which understanding can best grow.

Following this difference in goals, we can begin to identify cultural differences in the scripts teachers in each country use to generate their lessons. These different scripts are probably based on different assumptions about the role of problem solving in the lessons, about the way students learn from instruction, and about the proper role of the teacher.

Although the analyses are preliminary, there appears to be a clear distinction between the U.S. and German scripts, on the one hand, and the Japanese script, on the other. U.S. and German lessons tend to have two phases: an initial acquisition phase and a subsequent application phase. In the acquisition phase, the teacher demonstrates or explains how to solve an example problem. The explanation might be purely procedural (as most often happens in the United States) or may include development of concepts (more often the case in

Germany). Yet the goal in both countries is to teach students a method for solving the example problem(s). In the application phase, students practice solving examples on their own while the teacher helps individual students who are experiencing difficulty.

Japanese lessons appear to follow a different script. Whereas in U.S. and German lessons instruction comes first, followed by application, in Japanese lessons the order of activity is generally reversed. Problem solving comes first, followed by a time in which students reflect on the problem, share the solution methods they have generated, and jointly work to develop explicit understandings of the underlying mathematical concepts. While students in U.S. and German classrooms must follow their teachers as they lead students through the solution of example problems, Japanese students have a different job: to invent their own solutions, then reflect on those solutions in an attempt to increase understanding.

In addition to these differences in goals and scripts, we also find differences in the coherence of lessons in the three countries. The greatest differences are between U.S. lessons and Japanese lessons. U.S. lessons are less coherent than Japanese lessons if coherence is defined by several criteria: U.S. lessons are more frequently interrupted, both from outside the classroom and within; U.S. lessons contain more topics—within the same lesson—than Japanese lessons; and Japanese teachers are more likely to provide explicit links or connections between different parts of the same lesson.

Looking beyond the flow of the lessons, we also find cross-cultural differences in the kind of mathematical content that is presented in the lessons. When viewed in comparison to the content of lessons in the 41 TIMSS countries, the average eighth-grade U.S. lesson in the video sample deals with mathematics at the seventh-grade level by international standards, whereas in Japan the average level is ninth grade. The content of German lessons averages at the eighth-grade level.

The quality of the content also differs across countries. For example, most mathematics lessons include some mixture of concepts and applications of those concepts to solving problems. How concepts are presented, however, varies a great deal across countries. Concepts might simply be stated, as in "the Pythagorean theorem states that a² + b² = c²," or they might be developed and derived over the course of the lesson. More than three-fourths of German and Japanese teachers develop concepts when they include them in their lessons, compared with about one-fifth of U.S. teachers. None of the U.S. lessons include proofs, whereas 10 percent of German lessons and 53 percent of Japanese lessons include proofs.

Finally, as part of the video study, an independent group of U.S. college mathematics teachers evaluated the quality of mathematical content in a sample of the video lessons. They based their judgments on a detailed written description of the content that was altered for each lesson to disguise the country of origin (e.g., by deleting references to currency). They completed a number of in-depth analyses, the simplest of which involved making global judgments of the quality of each lesson's content on a three-point scale (low, medium, and high). (Quality was judged according to several criteria, including the coherence of the mathematical concepts across different parts of the lesson and the degree to which deductive reasoning was included.) Whereas 39 percent of the Japanese lessons and 28 percent of the German ones received the highest rating, none of the U.S. lessons received the highest rating. Eighty-nine percent of U.S. lessons received the lowest rating, compared with 11 percent of Japanese lessons.

When we examine the kind of work students engage in during the lessons, we find a strong resemblance between Germany and the United States, with Japan looking distinctly different. Three types of work were coded in the video study: practicing routine procedures, applying concepts to novel situations, and inventing new solution methods or thinking. Ninety-six percent of student working time in Germany and 90 percent in the United States is spent practicing routine procedures, compared with 41 percent in Japan. Japanese students spend the majority of their time inventing new solutions that require conceptual thinking about mathematics.

A great deal of effort has been put into the reform of mathematics teaching in the United States in recent years. Numerous documents—such as the National Council of Teachers of Mathematics' Curriculum and Evaluation Standards for School Mathematics (1989) and Professional Standards for Teaching Mathematics (1991)—encourage teachers to change the way they teach. There is great agreement, at least among mathematics educators, as to what desirable instruction should look like. Although most of the current ideas stated in such documents are not operationalized to the extent that they could be directly coded, it is possible to view some of the indicators developed in the video study in relation to these current ideas.

When the video data are viewed in this way, Japanese teachers, in certain respects, come closer to implementing the spirit of current ideas advanced by U.S. reformers than do U.S. teachers. For example, Japanese lessons include high-level mathematics, a clear focus on thinking and problem solving, and an emphasis on students deriving alternative solution methods and explaining their thinking. In other respects, though, Japanese lessons do not follow such reform guidelines. They include more lecturing and demonstration than even the more traditional U.S. lessons, and we never observed calculators being used in a Japanese classroom.

Regardless of whether or not Japanese classrooms share features of "reform" classrooms, it is quite clear that the typical U.S. classroom does not. Furthermore, the U.S. teachers, when asked if they were aware of current ideas about the best ways to teach mathematics, responded overwhelmingly in the affirmative. Seventy percent of the teachers claimed to be implementing such ideas in the very lesson that we videotaped. When asked to justify these claims, the U.S. teachers referred most often to surface features, such as the use of manipulatives or cooperative groups, rather than to the key point of the reform recommendations, which is to focus lessons on high-level mathematical thought. Although some teachers appear to have changed these surface-level characteristics of their teaching, the data collected for this study suggest that these changes have not affected the deeper cultural scripts from which teachers work.

Bearing in mind the preliminary nature of these findings, as well as the interpretations of the findings, we can, nevertheless, identify four key points:

• The content of U.S. mathematics classes requires less high-level thought than classes in Germany and Japan.
• U.S. mathematics teachers' typical goal is to teach students how to do something, while Japanese teachers' goal is to help them understand mathematical concepts.
• Japanese classes share many features called for by U.S. mathematics reforms, while U.S. classes are less likely to exhibit these features.
• Although most U.S. math teachers report familiarity with reform recommendations, relatively few apply the key points in their classrooms.