International Requirements for Sampling, Data Collection, and Response Rates

Return to Methodology and Technical Notes

The OECD required all participating education systems (countries and subnational regions) to adhere to the TALIS 2018 technical standards, which provided detailed information about the target population, sampling, response rates, translation, survey administration, and data submission. According to the standards, the population covered in each education system should consist of as many as possible of the eligible International Standard Classification of Education (ISCED) Level 2 teachers and school administrators. Developed by the United Nations Educational, Scientific and Cultural Organization (UNESCO), ISCED is used by countries to map education levels across countries and education systems. In the United States, ISCED Level 2 teachers are those of students in grades 7, 8, and 9 (also labeled lower secondary education for convenience). To provide valid estimates of teacher and principal characteristics, the sample of TALIS teachers had to be selected in a way that represented the full population of ISCED 2 teachers in each education system. The sample design for TALIS 2018 was a stratified systematic sample, with the school sampling probability proportional to the estimated number of ISCED 2 teachers within each school. Samples were drawn using a two-stage sampling process. In the first stage a sample of schools was drawn, and in the second stage a sample of teachers was drawn within each selected school. Statistics Canada (StatCan), one of the members of the TALIS consortium responsible for the design and implementation of TALIS internationally, drew the sample of schools for each education system.

A minimum sample size of 4,000 teachers from a minimum of 200 schools was required in each participating education system. Following the TALIS consortium guidelines, replacement schools were identified at the same time the TALIS sample was selected by designating as replacement schools the two neighboring schools in the sampling frame. The international guidelines specified that within schools, a sample of 20 teachers was to be selected in an equal probability sample unless fewer than 20 teachers were available (in which case all teachers were selected).

Each education system collected its own data following international guidelines and specifications. The technical standards required that eligible teachers were those teaching at least one ISCED Level 2 class, regardless of subject matter. School principals or head administrators of each sampled school were also asked to participate. School principal and teacher data were collected independently, so teacher eligibility was not dependent on principal participation (or vice versa).

The response-rate target was at least 75 percent of schools and at least 75 percent of teachers across the participating schools in each education system. A minimum participation rate of 50 percent of schools from the original school sample and 75 percent of schools after replacement, was required for an education system’s data to be included in the main international comparisons. Education systems were allowed to use replacement schools (selected during the sampling process) to increase the response rate as long as the 50 percent benchmark before replacement had been reached.

The data collected by each participating education system was adjudicated by the TALIS international consortium to ensure that the data met the TALIS technical standards for data collection. The principal and teacher data were adjudicated separately. For school-level data adjudication depended on only school data (the principal participation) and for teacher-level data, adjudication depended on only teacher data (50 percent of teachers in the school had to participate). Thus, for TALIS 2018, there are two datasets—a school dataset with data from the principal survey, and a teacher dataset with data for teachers, each defined by their respective participation standards.

Internationally, TALIS 2018 was designed to be as inclusive as possible. In-scope teachers, those comprising the international target population, were all classroom teachers teaching at least one ISCED Level 2 class and their school principals, and all subject matters are included. Out-of-scope teachers included:

• Teachers in schools for special education needs students and their principals.
• Substitute, emergency, or occasional teachers who are defined as teachers filling in on a temporary basis (no longer than six consecutive weeks) for a teacher who is still employed as either a full-time or part-time teacher at the school.
• Teachers teaching exclusively to adults who are defined as teachers teaching only to adults, whether the adult students follow a standard or an adapted curriculum.
• Teachers on long-term leave who are defined as teachers “on long term leave” who are absent and not expected to be back during the survey administration period.
• Teacher aides who are typically non-professional or paraprofessional staff supporting teachers in providing instruction to students.
• Pedagogical support staff who provide services to students to support the instructional program, such as guidance counselors or librarians.
• Health and social support staff who are health professionals such as doctors, nurses, psychiatrists, psychologists, occupational therapists, and social workers.

National target populations aim for maximum coverage of eligible teachers and schools. However, in some rare cases, for reasons of practicality, safety or economy (e.g. remote schools, unique demographic groups, types of schools, areas under civil unrest, natural catastrophe), the national survey population may be reduced. The U.S. population was not reduced in any way. (See Table A-3 for coverage rates for each country).