|WORLDWIDE STUDY OF CLASSROOMS WITH MORE THAN 60 COUNTRIES AND OTHER EDUCATION SYSTEMS PARTICIPATING
|TIMSS tests a variety of subject and content areas:
The Trends in International Mathematics and Science Study (TIMSS) is a study of classrooms across the country and around the world. The National Center for Education Statistics (NCES), in the Institute of Education Sciences at the U.S. Department of Education, is responsible for the implementation of TIMSS in the United States. Beginning in 1995 and every 4 years thereafter, TIMSS has provided participating countries and other education systems with an opportunity to measure students' progress in mathematics and science achievement. Studies of students, teachers, schools, curriculum, instruction, and policy issues are also carried out to understand the educational context in which learning takes place.
TIMSS represents the continuation of a long series of studies conducted by the International Association for the Evaluation of Educational Achievement (IEA). The IEA conducted its First International Mathematics Study (FIMS) in 1964 and the Second International Mathematics Study (SIMS) in 1980–82. The First and Second International Science Studies (FISS and SISS) were carried out in 1970–71 and 1983–84, respectively. Since the subjects of mathematics and science are related in many respects and since there is broad interest among countries and other education systems in students' abilities in both subjects, TIMSS began to be conducted as an integrated assessment of both mathematics and science.
In 1995, TIMSS collected data on grades 3 and 4 as well as grades 7 and 8, and the final grade of secondary school (called TIMSS Advanced, which assessed only the most advanced grade 12 students in the United States), with 45 countries and other education systems participating. In 1999, data were collected only for eighth-grade students, with 38 countries and other education systems participating. Also in 1999, TIMSS was renamed from "Third International Mathematics and Science Study" to "Trends in International Mathematics and Science Study." In 2003 , 2007, 2011, and 2015, TIMSS collected data at grades 4 and 8 from between 50 and 70 education systems in each administration, with differing numbers participating at each grade. In addition, TIMSS Advanced was conducted again internationally in 2008 and 2015, and measured advanced mathematics and physics achievement in the final year of secondary school across countries. The United States participated in the 1995 and 2015 TIMSS Advanced assessments. For a complete list of all countries and other education systems that participated in TIMSS in each administration by grade, see https://nces.ed.gov/surveys/international/participating.asp
In addition to the math and science assessments given to students, supplementary information is obtained through the use of student, teacher, and school questionnaires. In 1995 and 1999, further component studies were implemented, including benchmark and video studies.
The TIMSS 1999 Benchmarking Study included states and districts or consortia of districts from across the United States that chose to participate. Findings were used to assess comparative international standings and to evaluate mathematics and science programs in an international context.
The TIMSS Videotape Study was the first study to collect videotaped records of classroom instruction; representative samples of eighth-grade mathematics classes in 1995 and 1999, and science classes in 1999, were drawn and one lesson was videotaped in each of the participating classrooms. These additional component studies provided a more detailed context for understanding mathematics and science teaching and learning in the classroom.
The designers of TIMSS 1995 chose to focus on curriculum as a broad explanatory factor underlying student achievement, and curriculum was considered to have three strands—which are also three influencers of student achievement: what society would like to see taught (the intended curriculum), what is actually taught (the implemented curriculum), and what the students learn (the attained curriculum). This view was first conceptualized for SIMS.
The first strand (the intended curriculum), reflects society's goals for teaching and learning, which in turn reflects the ideals and traditions of the greater society, although somewhat constrained by the resources of the education system. The second strand (the implemented curriculum), or what is being taught in the classroom is inspired by the intended curriculum but directly influenced by: 1) teacher education, training and experience; 2) by the nature and organizational structure of the school; 3) teacher interaction with colleagues; and 4) the composition of the student body. The last strand (the attained curriculum), or what the students actually learn, is influenced by both the intended and implemented curriculum as well as by the characteristics of individual students--such as abilities, attitudes, interests, and effort.
Because TIMSS 1999 essentially replicated the eighth-grade part of the 1995 study, much of the conceptual underpinnings of the 1999 and subsequent-year TIMSS studies are derived from this three-strand model of curriculum. The organization and coverage of the intended curriculum (first strand) was investigated through curriculum questionnaires that were completed by national research coordinators (NRCs) and by curriculum advisors. Although more modest in scope than the extensive curriculum analysis component of 1995, the 1999 and subsequent-year TIMSS questionnaires yield valuable information on the curricular intentions of participating countries and other education systems.
Data on the implemented curriculum (second strand) were collected as part of the 1999 and subsequent-year TIMSS surveys of student achievement. Questionnaires completed by the mathematics and science teachers of the students in the survey, and by the principals of their schools, provide information about the topics in mathematics and science that were taught, the instructional methods used in the classroom, the organizational structures that supported teaching, and the factors that were observed that may have facilitated or inhibited teaching and learning.
The student achievement survey provides information for the third strand, the attained curriculum. The wide-ranging mathematics and science tests that were administered to nationally representative samples of students provided not only a sound basis for international comparisons of student achievement, but a rich resource for the study of the attained curriculum in each country or other education system. Information about the student's characteristics, attitudes, beliefs, and experiences was collected from each participating student and helped to identify characteristics associated with learning and to provide a context for the attained curriculum.
TIMSS is designed to measure student performance in mathematics and science against what is expected to be taught in school. This focus on school curriculum allows for two broad questions to be addressed through TIMSS: (1) How do mathematics and science educational environments and student outcomes differ across countries and other education systems, and how are the differences in student outcomes related to the differences in educational environments? (2) Are there patterns of relationships among contexts, inputs, and outcomes within countries and other education systems that can lead to improvements in the theories and practices of mathematics and science education?
TIMSS uses several types of instruments to collect data about students, teachers, schools, and national policies and practices that may contribute to student performance.
Written Assessment. Assessments are developed to test students in various content areas within mathematics and science. For grade 4, the mathematics content areas are numbers; geometric shapes and measures; and data display. The grade 4 science content areas are Earth science; life science; and physical science. The grade 8 mathematics content areas are numbers; algebra; geometry; and data and chance. The grade 8 science content areas are biology; physics; chemistry; and Earth science.
In addition to being familiar with the mathematics and science content areas encountered in TIMSS, students are required to draw on a range of cognitive skills to successfully complete the assessment. TIMSS focuses on three cognitive domains in each subject: knowing, which covers the facts, procedures, and concepts students need to know; applying, which focuses on the ability of students to apply their knowledge and conceptual understanding to solve problems; and reasoning, which goes beyond solving routine problems to include unfamiliar situations and context that may require multi-step problem-solving.
After each TIMSS assessment cycle, approximately half of the items are publicly released, and replacement items that closely match the content of the original items are developed by international assessment and content experts. These new items are field tested and refined to the point where a variety of multiple choice and extended constructed-response items (i.e., items requiring written explanations from students) are chosen to be included in the TIMSS item pool.
Each student is asked to complete one booklet, made up of a subset of items taken from this item pool. No student answers all of the items in the item pool. The scoring of these booklets is accomplished through the use of a sophisticated and strict set of criteria that are implemented equally across all nations to ensure accuracy and comparability.
Student Background Questionnaire. Each student who takes the TIMSS assessment is asked to complete a questionnaire on issues including daily activities, family attributes, educational resources in the home, engagement in and beliefs about learning, instructional processes in the classroom, study habits, and homework.
Teacher Questionnaire. The teacher questionnaire is given to the mathematics and science teachers of the students assessed in the study. These questionnaires ask about topics such as attitudes and beliefs about teaching and learning, teaching assignments, class size and organization, topics covered in class, the use of various teaching tools, instructional practices, professional preparation, and continuing development.
The teacher questionnaire is designed to provide information about the teachers of the students in the TIMSS student samples. The teachers who complete TIMSS questionnaires do not constitute a sample from any definable population of teachers. Rather, they represent the teachers of a national sample of students.
School Questionnaire. The principal or head administrator is also asked to complete a questionnaire for the school focused on community attributes, personnel, teaching assignments, policy and budget responsibilities, curriculum, enrollment, student behavior issues, instructional organization, and mathematics and science courses offered.
Information collected from students, their teachers and schools is summarized in composite indices focused, in particular, on the relationship between mathematics and science achievement and the home, classroom, and school environment.
Curriculum Questionnaire. The NRC, or representative, of each participating country or other education system is asked to complete a questionnaire focused on the policies and practices supported at the national level that may contribute to student performance. In addition, because the mathematics and science topics covered in the assessment may not be included in all countries and other education systems' curriculum, the NRCs are asked to indicate whether each topic covered in TIMSS is included in their countries and other education systems' intended curriculum through the fourth or eighth grade.
Encyclopedia. Beginning with TIMSS 2007, each participating country or other education system is asked to provide a written overview of the context in which mathematics and science instruction takes place, summarizing the structure of the education system, the mathematics and science curricula and instruction in primary and secondary grades, teacher education requirements, and the types of examinations and assessments employed to monitor success. The resulting chapters are compiled in a publication entitled the TIMSS Encyclopedia.
Videotape Study. The 1995 TIMSS Videotape Study was designed as the first study to collect videotaped records of classroom instruction from national probability samples in Japan, Germany, and the United States in order to gather more in-depth information about the context in which learning takes place as well as to enhance understanding of the statistical indicators available from the main TIMSS study. An hour of regular classroom instruction was videotaped in a subsample of eighth-grade mathematics classrooms (except in Japan, where videotaping was usually done in a different class, selected by the principal) included in the assessment phase of TIMSS in each of the three countries.
The 1999 TIMSS Videotape Study was expanded in scope to examine national samples of eighth-grade mathematics and science instructional practices in seven nations: Australia, the Czech Republic, Hong Kong, Japan, the Netherlands, Switzerland, and the United States. Four countries—Australia, the Czech Republic, the Netherlands, and the United States—participated in both the mathematics and science components of the study. Hong Kong and Switzerland participated in only the mathematics component, and Japan in only the science component.
Ethnographic Case Studies. The case studies approach to understanding cultural differences in behavior has a long history in selected social science fields. Conducted only in 1995, the case studies were designed to focus on four key topics that challenge U.S. policymakers and to investigate how these topics were dealt with in the United States, Japan, and Germany: implementation of national standards; the working environment and training of teachers; methods for dealing with differences in ability; and the role of school in adolescents' lives. Each topic was studied through interviews with a broad spectrum of students, parents, teachers, and educational specialists. The ethnographic approach permitted researchers to explore the topics in a naturalistic manner and to pursue them in greater or lesser detail, depending on the course of the discussion. As such, these studies both validated and integrated the information gained from official sources with that obtained from teachers, students, and parents in order to ascertain the degree to which official policy reflected actual practice. The objective was to describe policies and practices in the nations under study that were similar to, different from, or nonexistent in the United States.
In three regions in each of the three countries, the re-search plan called for each of the four topics to be studied in the fourth, eighth, and twelfth grades. The specific cities and schools were selected "purposively" to represent different geographical regions, policy environments, and ethnic and socioeconomic backgrounds. Schools in the case studies were separated from schools in the main TIMSS sample. Where possible, a shortened form of the TIMSS test was administered to the students in the selected schools. The ethnographic researchers in each of the countries conducted interviews and obtained information through observations in schools and homes. Both native-born and nonnative researchers participated in the study to ensure a range of perspectives.
TIMSS Benchmarking Study. In 1999, 13 states and 14 districts or consortia of districts throughout the United States participated as their own "nations" in this project, following the same guidelines as the participating countries and other education systems. The samples drawn for each of these states and districts were representative of the student population in each of these states and districts. The findings from this project allowed these jurisdictions to assess their comparative international standing and judge their mathematics and science programs in an international context.
In 2003, 2007, 2011, and 2015, several states again participated in TIMSS to benchmark their student performance internationally. In 2003, Indiana participated in TIMSS independently as well as part of the U.S. national sample; in 2007, Massachusetts and Minnesota did; and in 2011, Alabama, California, Colorado, Connecticut, Florida, Indiana, Massachusetts, Minnesota, and North Carolina did. In 2015, Florida participated as an international benchmark.
NAEP/TIMSS linking studies. NCES conducted a study in 2012 to link results from the 2011 National Assessment of Educational Progress (NAEP) to results from the 2011 TIMSS assessment. The goal of the NAEP-TIMSS Linking Study was to predict 2011 TIMSS mathematics and science scores at grade 8 for all U.S. states based on their NAEP performance for states, without incurring the costs associated with every state participating in TIMSS. The 2011 NAEP-TIMSS Linking Study represents the fourth study to link NAEP and TIMSS. The first study used results from the 1995 administration of TIMSS and the results from the 1996 NAEP. The second study used results from TIMSS 1999 and NAEP 2000. The third study examined linking 2007 NAEP and TIMSS data through the method of statistical moderation, by applying the correspondence between the national score distributions of NAEP and TIMSS to the state and district NAEP score distributions. The 2011 NAEP-TIMSS Linking Study was undertaken to improve on the previous attempts to link these two assessments. The 2011 NAEP-TIMSS Linking Study differed from the previous linking studies in several ways. First, unlike previous efforts, the 2011 NAEP-TIMSS Linking Study administered NAEP and TIMSS booklets at the same time under the same testing conditions. This provided examinee-level correlations between NAEP and TIMSS scores to improve the accuracy of the predicted TIMSS state scores. The second way that it differed is that a greater number of validation states, nine, participated in TIMSS, which meant a greater number of actual state TIMSS results were available to validate the predicted TIMSS average scores.
To validate the 2011 NAEP-TIMSS Linking Study, nine independent state samples were used from Alabama, California, Colorado, Connecticut, Florida, Indiana, Massachusetts, Minnesota, and North Carolina.
TIMSS Advanced studies. In 1995, TIMSS Advanced was first conducted to determine what students, who were finishing secondary school with special preparation in mathematics and science, know and can do. The achievement of these students was and continues to be of great interest to policy makers, as it may help determine a country's potential in global competitiveness. TIMSS Advanced has been repeated twice since 1995, in 2008 and 2015. The goal of TIMSS Advanced is to compare the performance of the most advanced final-year secondary students across participating countries in advanced mathematics and physics. The advanced mathematics content includes calculus; numbers, equations, and functions; validation and structure; probability and statistics; and geometry. The physics assessment includes questions about mechanics; electricity and magnetism; particle, quantum and other types of modern physics; heat; and wave phenomena. Sixteen countries participated in the 1995 TIMSS Advanced assessment; In the United States, the advanced mathematics assessment in 1995 was administered to a sample of 12th-grade students who had taken or were taking pre-calculus, calculus, AP calculus, or calculus and analytic geometry. The physics assessment in 1995 was administered to a sample of 12th-grade U.S. students who had taken or were taking at least one year-long course in physics, including physics I, physics II, advanced physics, and Advanced Placement physics. Nine education systems—all IEA member countries—participated in TIMSS Advanced 2015. In the United States, the advanced mathematics sample in 2015 was restricted to 12th-grade students who had taken or were taking a class teaching calculus to be more internationally comparable. The physics sample in 2015 was restricted to 12th-grade U.S. students who had taken or were taking a second-year physics class, an AP Physics course, or the equivalent. For additional information on TIMSS Advanced 1995 and 2015, please see U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User's Guide (Averett et al. 2017).
First conducted in the spring of 1995, TIMSS has been conducted every 4 years since then. TIMSS assessments are administered near the end of the school year in each country or other education system. In countries and other education systems in the Southern Hemisphere (where the school year typically ends in November or December) the assessment is conducted in October or November. In the Northern Hemisphere, the school year typically ends in June; so in these countries and other education systems the assessment is conducted in April, May, or June.
The U.S. National and International TIMSS data files are available at https://nces.ed.gov/timss/datafiles.asp. The TIMSS International Database, its user's guide and supplements, and supporting materials are available for download for each round of TIMSS from the TIMSS and PIRLS International Study Center website at https://timssandpirls.bc.edu/?_sm_au_=iMVrFHZZ1jJvfVTq.