In addition to the following questions about TIMSS, more FAQs about international assessments are available at: http://nces.ed.gov/surveys/international/faqs.asp.
The TIMSS 1999 Video Study examines teaching practices in the United States and six other relatively high-achieving countries as measured on the TIMSS 1995 mathematics assessment (Australia, the Czech Republic, Hong Kong SAR, Japan, the Netherlands, and Switzerland). Its novel approach in which eighth-grade mathematics and science lessons are videotaped allows for in-depth investigation of teaching as it is actually experienced by students in the classroom. The TIMSS 1999 Video Study is an update and expansion of the TIMSS 1995 Video Study, which examined videotapes of eighth-grade mathematics lessons in Germany, Japan, and the United States.
Click here for more info on the TIMSS 1999 Video Study
Traditionally, attempts to measure classroom teaching have used teacher questionnaires. Although questionnaires are relatively easy to administer on a large scale and can provide useful quantitative information, they rely on a teacher's memory and conscious behaviors. Moreover, questions are subject to a teacher's interpretation and cannot be revisited once the questionnaire has been completed. The benefits of a video study are as follows:
Depending on the school-year calendar in each participating country, data collection began in 1998 and continued through 2000. Sampling procedures required that lessons were randomly selected to be representative of eighth-grade mathematics and science lessons overall in each participating country. This includes eighth-grade mathematics and science lessons collected in Japan in 1995 as part of the prior video study. In each case, a teacher was videotaped for one complete, regular lesson (lessons devoted entirely to testing were not included). Lessons were selected across the year to try to capture the range of topics and activities that can take place throughout an entire school year. Schools were not permitted to substitute classes for the ones randomly selected. Finally, the sample from each country was reviewed for consistency by an international sampling expert. No substitutions of schools, teachers, students, or classes were allowed.
Once a lesson had been videotaped, the lesson was sent to a central processing unit where the videos were carefully tracked. The first step in the process was to check the quality of the videotaping to ensure that videographers followed the established filming procedures. Videotapes were then transcribed and translated using a state-of-the-art video processing software. Each transcription and translation was checked for accuracy. Each utterance or event in a lesson was then associated with a time point in the lesson to allow for analyses such as the duration of a lesson and the length of time spent reviewing homework. An international team of mathematics and education experts developed a set of codes to apply to the 1999 video data, initially based on the TIMSS 1995 Video Study but greatly expanded and improved. The codes were carefully reviewed by the national research coordinator in each participating country as well as a technical review panel assembled to assist with the study. The quality and reliability of the coding process was monitored carefully throughout the year-long process by calculating inter-rater reliability as well as the reliability of codes within and across countries. Coders were trained extensively before the coding process began, and were retrained when it was determined that they did not meet the minimum reliability established for the study. Coders were monitored carefully throughout the coding process to avoid reliability decay. Codes were dropped from the study if 85 percent reliability could not be achieved (or if individual coders could not reach at least 80 percent reliability). All codes presented in reports met or exceeded the minimum acceptable reliability standard established for the study.
In general, the sampling plan for the TIMSS 1999 Video Study followed the standards and procedures agreed to and implemented for the TIMSS 1999 assessments. The school sample was required to be a Probability Proportionate to Size (PPS) sample. A PPS sample assigns probabilities of selection to each school proportional to the number of eligible students in the eighth -grade in schools countrywide. Then, one mathematics and/or one science eighth-grade class per school was sampled, depending on the subject(s) to be studied in each country. Only one class was randomly selected within each school. No substitutions of teachers or class periods were allowed. The designated class was videotaped once, in its entirety, without regard to the particular mathematics topic being taught or type of activity taking place. Teachers were asked to not prepare special lessons for the videotaping; rather, they were asked to conduct the lessons they would normally have given on the day of videotaping, and to deliver it in the way they would normally as if they were not being videotaped. The only exception was that teachers were not videotaped on days they planned to give a test for the entire class period. The final sample of classes captured on videotape was reviewed by an international sampling referee. Weights were developed for each country's data to account for sampling error and to allow for appropriate national estimates. Finally, to determine whether the final sample of teachers included in the study were generally representative of teachers of eighth-grade classes in their respective countries, the teachers were asked a series of questions, such as their academic preparation, years of teaching experience, certification status and teaching load, that could be cross-referenced with other known data sets to determine the representativeness of the final sample. Analysis of the data indicate that, in general, the teachers selected for inclusion in the study in each country are similar to the larger population of teachers who teach at that grade level.
Being videotaped could have affected the typicality and quality of the lessons. Teachers were asked specifically about the influence of the camera in the classroom, both in terms of the influence on their teaching and on students' behavior. When teachers were asked whether the camera caused them to teach a lesson that was worse than usual, same as usual, or better than usual, the majority of lessons were taught by teachers who rated their lesson as about the same as usual, except in the case of the Czech Republic. An equal percentage of lessons in the Czech Republic were taught by teachers who rated their lesson as about the same or worse than usual. In terms of students' behavior during the videotaped lessons, a majority of lessons were taught by teachers who reported their students' behavior as being about the same or better than usual.
Teachers were asked to do nothing special for the videotape session, and to conduct the class as they had planned. The scheduler and videographer in each country determined on which day the lesson would be filmed. Following agreed upon sampling procedures, videographers were sent to each country to capture on tape lessons as they were implemented in the classroom. To determine whether teachers conducted special lessons, they were asked a series of questions through the teacher questionnaire. Among the questions were those that asked whether the videotaped lesson was part of a larger unit of lessons or a stand-alone lesson; whether the lesson was one that had been prepared especially for that day or had been used before; how long they spent preparing for the lesson in comparison to the amount of time they normally spent preparing; whether the teaching methods implemented in the videotaped lesson were typical or unusual from their normal teaching habits; and the like. The analyses reveal that most eighth-grade lessons captured in each country were taught by teachers who considered the videotaped lesson to be typical of their teaching.
The broad purpose of the TIMSS 1999 Video Study was to investigate and describe teaching practices in eighth-grade mathematics and science in a variety of countries. The goal of the study was not to identify the key features of "good" or "exemplary" teaching, nor was it designed to identify master teachers. Rather, it was important to capture the kind of teaching that students experience on a daily basis in their mathematics and science classes. The results of this study make it clear that an international comparison of teaching, even among mostly high achieving countries, cannot, by itself, yield a clear answer to the question of which method of teaching may be best to implement in a given country. Interpreting the results from this study requires a thoughtful and analytic approach, including follow-up analyses and research that can more precisely examine the possible effects that particular methods or approaches may have on student learning. Through these kinds of activities, the ultimate aim of a study such as this can be realized: a deeper understanding of classroom mathematics teaching and a deeper understanding of how teaching methods can be increasingly aligned with learning goals for students.
The videotaped lessons upon which analyses were conducted and reported were collected with the understanding that they would never be shown publicly. This is due largely to privacy and confidentiality concerns. However, the study sponsors and researchers recognized the value in making lessons from each country available to public to provide examples of the many types of activities and behaviors observed in the main database. To address this anticipated need, additional lessons were collected in each country that were specifically intended for public viewing. Included with the full report is a CD-ROM that contains video clip examples taken from the public-use videos from each country. Some of the video clips are also available for viewing in the highlights of the report. In addition, the full-length versions of the public-use lesson are available for purchase at cost, from two sources: LessonLab and Research for Better Schools.
The reason for conducting a study of teaching is straightforward: to better understand, and ultimately improve, students' learning, one must examine what happens in the classroom. Although relationships between classroom teaching and learning are complicated, it is well documented that teaching makes a difference in students' learning. Observing that teaching influences students' learning is not the same as claiming that teaching is the sole cause of students' learning, however. Many factors, both inside and outside of school, can affect students' levels of achievement. In particular, eighth-graders' achievement is the culmination of many past and current factors. For these reasons, no direct inferences can or should be made to link descriptions of teaching in the TIMSS 1999 Video Study with students' levels of achievement as documented in TIMSS 1999. Moreover, in most of the participating countries, the videotaped classrooms were not the same ones in which students took the achievement tests.
One of the key hypotheses that emerged from the 1995 Video Study was that different countries have distinct cultural patterns of teaching. With this in mind, the 1999 Video Study was designed with a coding system that captured features considered essential from each country's perspective. The 1999 study also expanded the number of countries studied in order to compare teaching practices in multiple high-achieving nations, in particular, those that outperformed the United States on TIMSS. The 1999 study built upon the coding scheme developed for the 1995 study, expanding and further developing its key ideas.