How Were the Videotaped Lessons Analyzed?

Three teams were assembled to develop and apply codes that would capture teaching activities and behaviors observed in the videotaped lessons. The Science Code Development Team (mentioned previously) included science specialists, researchers, and representatives from each of the participating countries who identified and developed codes, trained coders and established reliability, organized quality control measures, and managed the analyses and reporting of the data. The Science Code Development Team worked closely with two advisory groups consisting of national research coordinators representing each of the countries in the study and a steering committee of five U.S. science education researchers. The International Video Coding Team represented all of the participating countries and applied the developed codes to each of the videotaped lessons. The Science Content Coding Team included U.S. representatives with expertise in science content who developed and applied codes to all of the scientific content of the videotaped lessons.

Extensive training was conducted for the International Video Coding Team and the Science Content Coding Team. Reliability was established for codes that identified an activity or behavior and measured how long the activity or behavior took place. For certain codes, the members of the Science Content Coding Team each established reliability through consensus coding.

Finally, to obtain reliable comparisons among the participating nations, the data were appropriately weighted to account for sampling design.

Comparisons made in this document have been tested for statistical significance at the .05 level. Differences between averages or percentages that are statistically significant are discussed using comparative terms such as “higher” or “lower.” Moreover, these differences are noted using the greater than symbol (>) in the footnotes of each table or figure included in this document. Differences that are not statistically significant are either not discussed or referred to as “no measurable differences found” or “not statistically significant.” Failure to find a statistically significant difference should not be interpreted to mean that the estimates are the same or similar; rather, failure to find a difference may also be due to measurement or sampling error.

The purpose of this report is to introduce new NCES survey data through the presentation of selected descriptive information. Readers are cautioned not to draw causal inferences based solely on the bivariate results presented. It is important to note that many of the variables examined in this report are related to one another, and complex interactions and relationships have not been explored here. Release of the report is intended to make the information available to the public and encourage more in-depth analysis of the data.

Limitations in an international video study of this type are related to the intensive and extensive nature of the data collection and coding processes. For example, a limited number of countries, and classrooms within the countries, were included in the TIMSS 1999 Video Study. As a result, the sample of four relatively higher-achieving countries in this study may not be representative of all the countries with students performing well on international assessments of science. In addition, the TIMSS 1999 Video Study cannot address differences in topic-specific teaching approaches because a wide range of topics were covered; not enough lessons covering any single topic were observed to allow within-topic analyses or comparisons across topics. Because the focus of the TIMSS 1999 Video Study is on recording and interpreting a complex set of teaching practices, it does not address students’ classroom behaviors and other characteristics of students. Finally, the reader is cautioned that direct inferences about links between classroom teaching and student achievement cannot be drawn from a study of this type.

Top