This section describes features of the Teaching and Learning International Survey (TALIS) 2018 methodology including sample design and survey design with a particular focus on the U.S. implementation. For further details about the assessment and any of the topics discussed here, see the Organization for Economic Cooperation and Development (OECD)’s TALIS 2018 Technical Report (OECD 2019).
The OECD required all participating education systems (countries and subnational regions) to adhere to the TALIS 2018 technical standards, which provided detailed information about the target population, sampling, response rates, translation, survey administration, and data submission. According to the standards, the population covered in each education system should consist of as many as possible of the eligible International Standard Classification of Education (ISCED) Level 2 teachers and school administrators. Developed by the United Nations Educational, Scientific and Cultural Organization (UNESCO), ISCED is used by countries to map education levels across countries and education systems. In the United States, ISCED Level 2 teachers are those of students in grades 7, 8, and 9 (also labeled lower secondary education for convenience). To provide valid estimates of teacher and principal characteristics, the sample of TALIS teachers had to be selected in a way that represented the full population of ISCED 2 teachers in each education system. The sample design for TALIS 2018 was a stratified systematic sample, with the school sampling probability proportional to the estimated number of ISCED 2 teachers within each school. Samples were drawn using a two-stage sampling process. In the first stage a sample of schools was drawn, and in the second stage a sample of teachers was drawn within each selected school. Statistics Canada (StatCan), one of the members of the TALIS consortium responsible for the design and implementation of TALIS internationally, drew the sample of schools for each education system.
A minimum sample size of 4,000 teachers from a minimum of 200 schools was required in each participating education system. Following the TALIS consortium guidelines, replacement schools were identified at the same time the TALIS sample was selected by designating as replacement schools the two neighboring schools in the sampling frame. The international guidelines specified that within schools, a sample of 20 teachers was to be selected in an equal probability sample unless fewer than 20 teachers were available (in which case all teachers were selected).
Each education system collected its own data following international guidelines and specifications. The technical standards required that eligible teachers were those teaching at least one ISCED Level 2 class, regardless of subject matter. School principals or head administrators of each sampled school were also asked to participate. School principal and teacher data were collected independently, so teacher eligibility was not dependent on principal participation (or vice versa).
The response-rate target was at least 75 percent of schools and at least 75 percent of teachers across the participating schools in each education system. A minimum participation rate of 50 percent of schools from the original school sample and 75 percent of schools after replacement, was required for an education system’s data to be included in the main international comparisons. Education systems were allowed to use replacement schools (selected during the sampling process) to increase the response rate as long as the 50 percent benchmark before replacement had been reached.
The data collected by each participating education system was adjudicated by the TALIS international consortium to ensure that the data met the TALIS technical standards for data collection. The principal and teacher data were adjudicated separately. For school-level data adjudication depended on only school data (the principal participation) and for teacher-level data, adjudication depended on only teacher data (50 percent of teachers in the school had to participate). Thus, for TALIS 2018, there are two datasets—a school dataset with data from the principal survey, and a teacher dataset with data for teachers, each defined by their respective participation standards.
Internationally, TALIS 2018 was designed to be as inclusive as possible. In-scope teachers, those comprising the international target population, were all classroom teachers teaching at least one ISCED Level 2 class and their school principals, and all subject matters are included. Out-of-scope teachers included:
National target populations aim for maximum coverage of eligible teachers and schools. However, in some rare cases, for reasons of practicality, safety or economy (e.g., remote schools, unique demographic groups, types of schools, areas under civil unrest, natural catastrophe), the national survey population may be reduced. The U.S. population was not reduced in any way. (See Table A-3 for coverage rates for each country.)
The TALIS 2018 school sample of 220 schools was drawn for the United States by the TALIS consortium. The U.S. sample was stratified into five explicit groups based on school control (public/private) and grade structure (with three groups—middle/junior high schools with grades 6–8 or 7–9, high schools with grades 9–12, or other schools with grade structures that include at least one ISCED 2 grade). Given the small number of private schools with a middle/junior high school grade structure, this stratum was collapsed with the high school grade structure in private schools. Within each explicit stratum, schools were sorted by census region (Northeast, Midwest, South, and West), locale (urban/suburban/town-rural),1 percent minority students,2 state, and number of ISCED 2 students (measure of size—see paragraph below).
The school sampling frame included any school containing at least one of grades 7 through 9 (defined as ISCED 2 in the United States). The data for public schools were from the 2015–16 Common Core of Data (CCD) and the data for private schools were from the 2015–16 Private School Universe Survey (PSS). The U.S. TALIS 2018 national school sample consisted of 220 schools. This number represents an increase from the international minimum requirement of 200 and was implemented to offset anticipated school nonresponse and reduce design effects.
The CCD and PSS databases include estimates of the number of teachers per school. While the number of full-time equivalent teachers (FTE) was available from the CCD, the number of teachers by ISCED level was not. Following the advice of the TALIS consortium, Westat, a research organization under contract to the National Center for Education Statistics (NCES) in the U.S. Department of Education, used the number of estimated ISCED 2 students from the CCD and PSS databases as the measure of size. Student estimates rather than teachers were used due to the uncertainty of the number of ISCED 2 teachers in certain types of schools (e.g., grade 9 teachers in grades 9–12 high schools).
Overlap with the 2018 National Teacher and Principal Survey (NTPS) school sample was minimized for the 2018 TALIS. The NTPS school frame was developed from the 2014–15 CCD. The school sample for the NTPS was selected by May 18, 2017, by the U.S. Census Bureau, and the Census Bureau provided Westat with the complete frame of schools (public schools only) with NTPS probabilities of selection for every school on their frame. Westat provided Statistics Canada the NTPS probabilities for the schools that matched the TALIS frame, and in drawing the U.S. TALIS sample Statistics Canada used this information of minimize overlap with NTPS.
In order to obtain a sample of teachers within schools, participating schools provided a list of TALIS eligible teachers (typically in January and February of 2018), and with the data from this list the sample was drawn using sampling software provided by the international contractor.
1 Town and rural were combined to create three equal-sized groups because the category “town” was much smaller than the other three categories.
2 The percent minority students is defined as “15 percent and above” or “below 15 percent” Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native, and l students of Two or more races.
The 2018 survey instruments were developed to cover a set of predefined themes defined by the TALIS framework (Ainley and Carstens 2018). These themes were identified by representatives of the participating education systems, the Questionnaire Expert Group (QEG), and the OECD. Items were reviewed by the TALIS QEG and by representatives of each country for their relevance to TALIS’s goals and for possible biases. All participating education systems field-tested the survey items in spring 2017. After the field trial, the descriptive statistics and psychometric properties of the items were reviewed, and items that did not meet the established measurement criteria were dropped for the main assessment.
The field trial also served as a means to test the field operations for the TALIS survey. The field trial evaluated the efficiency, accuracy, and effectiveness of the online questionnaire system in capturing information.
The final U.S. 2018 main study principal survey included 48 questions and the U.S. 2018 teacher survey included 55 questions. Countries were permitted to add “national only” questions/answers and answer categories. Also, each country adapted the international questionnaire to fit national terms, definitions, spelling, and punctuation, which is explained further in the following section. The resulting final set of items covered all 9 content themes of the framework. The United States added two questions to both the principal and teacher surveys asking about the principal’s/teacher’s race and ethnicity. The U.S. principal survey also added a question asking principals to rate a series of educational goals. The final national principal and teacher surveys are available online at: https://nces.ed.gov/surveys/talis/questionnaire.asp.
International source versions of the principal and teacher surveys and school-level manuals were prepared in English and French and translated into the primary language or languages of instruction in each education system. The TALIS consortium recommended that each education system follow a double translation design and provided precise translation guidelines that included a description of the features each item was measuring and for the main study statistical analysis from the field trial. These translation procedures entailed having two independent translations, one from each of the source languages (English and French), with differences reconciled by a third party. When double translation was not possible, single translation was accepted.
Instrument adaptation was necessary even in nations such as the United States that use English as the primary language of instruction. These adaptations were primarily for cultural purposes. For example, words such as “lift” might be adapted to “elevator” for the United States. The TALIS consortium verified and approved the national adaptation of all instruments, including those of the United States.
The TALIS 2018 school recruitment strategy included: (1) starting recruitment at the beginning of the school year in 2017; (2) incorporating assistance from National Assessment of Educational Progress (NAEP) State Coordinators within the states with sampled TALIS schools or approaching schools directly, as well as sending information to relevant school districts and states; and (3) providing cash incentives to both schools and teachers.
Westat staff initiated school recruitment activities in September 2017. Contact with states and districts began through a mailing in late September 2017, after an initial virtual meeting with NAEP State Coordinators. Schools were contacted through an initial mailing shortly after the district mailing. These initial mailings were sent from the NAEP State Coordinators and included a letter from NCES, or the state chief, addressed to the appropriate district or school official, and a folder of TALIS materials that contained a study brochure, timeline, and a list of frequently asked questions (FAQs). Twenty-eight school districts required the formal review and approval of a research proposal before schools could be contacted. Formal research requests were prepared and sent to these districts. In those states where NAEP State Coordinators were assisting with school recruitment, NAEP State Coordinators indicated when follow-up contacts could be made by Westat field staff. In states where the NAEP State Coordinators were not assisting with school recruitment, Westat field staff contacted districts and schools independently. Schools were asked to identify school coordinators to facilitate the survey within the schools and liaise with Westat during the TALIS data collection. School coordinators were asked to prepare a list of eligible teachers for the within-school teacher sample, distribute paper instruments, track teacher and principal participation, and distribute teacher incentives. The school coordinators of participating schools were offered $200, teachers were offered $20 to complete the teacher questionnaire, and the schools were offered $200 for participating. The data collection window for TALIS began March 1, 2018 and ended May 31, 2018.
The TALIS consortium emphasized the use of standardized procedures in all education systems. Each education system collected its own data, based on detailed manuals provided by the TALIS consortium that explained the survey’s implementation including precise instructions for the work of school coordinators. Survey administration in the United States was performed primarily using an online instrument with paper-based surveys also available. The paper instruments were provided upon request and also sent to nonresponding teachers and principals to prompt their response.
In each education system, a TALIS International Quality Observer (IQO) who was contracted independently by the TALIS consortium reviewed survey administrations in a subsample of participating schools. The schools in which the independent observations were conducted were selected jointly by the TALIS consortium and the IQO. In the United States, there was one IQO who observed 24 schools from the national sample. The IQO’s primary responsibility was to document the extent to which survey procedures in schools were implemented in accordance with administration procedures. The IQO’s observations in U.S. schools indicated that international procedures for data collection were applied consistently.
The use of sampling weights is necessary for computing statistically sound nationally representative estimates. Survey weights were adjusted for the probabilities of selection for individual schools and teachers, for school or teacher nonresponse, and for errors at the time of sampling in estimating the size of the school or the number of eligible teachers in the school. Survey weighting for all education systems participating in TALIS 2018 was coordinated by the international TALIS consortium.
The school base weight was defined as the reciprocal of the school’s probability of selection. (For replacement schools, the school base weight was set equal to the original school it replaced.) Within each explicit stratum, the school base weights were then adjusted for nonresponding schools that were not replaced by replacement schools to produce the final school weights.
The teacher weighting was more complicated than the school weighting because in addition to the teacher base weight it included the school base weight as well as four additional adjustment factors: school nonresponse, teacher nonresponse, incidental inclusions, and multiplicity. The teacher base weight was defined as the reciprocal of the within school probability of selection for each selected teacher. The teacher nonresponse adjustment was an adjustment that allocates the weight of the nonresponding teachers to responding teachers within each school. The teacher nonresponse adjustment included adjustments that accounted for nonresponding teachers as well as teachers that left the school after the teachers were selected for the sample. The incidental inclusion adjustment adjusts the teacher weight for teachers who were counted originally but then excluded because they were also principals. The final weighting adjustment was the teacher multiplicity factor that adjusted the weights for teachers working in more than one school. For most teachers, the adjustment factor was 1; for the others, it was the reciprocal of the number of schools in which they taught. The final teacher weights were the product of the final school weight, the teacher base weight, and the three teacher adjustment factors.
All TALIS analyses were conducted using these adjusted sampling weights. For more information on the nonresponse adjustments, see OECD’s TALIS 2018 Technical Report (OECD 2019).
As with any study, there are limitations to TALIS 2018 that should be taken into consideration. Estimates produced using data from TALIS 2018 are subject to two types of error: non-sampling errors and sampling errors.
Non-sampling error is a term used to describe variations in the estimates that may be caused by population coverage limitations, nonresponse bias, and measurement error, as well as data collection, processing, and reporting procedures. For example, suppose the study was unsuccessful in getting permission from many rural schools in a certain region of the country. In that case reports of means for rural schools for that region may be biased. The sources of non-sampling errors are typically problems such as item nonresponse, the differences in respondents’ interpretations of the meaning of survey questions, and mistakes in data preparation, such as miskeying paper survey responses or errors in data cleaning.
Sampling errors arise when a sample of the population, rather than the whole population, is used to estimate some statistic. Different samples from the same population would likely produce somewhat different estimates of the statistic in question. This means that there is a degree of uncertainty associated with statistics estimated from a sample. This uncertainty is referred to as sampling variance and is usually expressed as the standard error of a statistic estimated from sample data. The approach used for calculating standard errors in TALIS is the Fay method of balanced repeated replication (BRR) (Judkins 1990). This method of producing standard errors uses information about the sample design to produce more accurate standard errors than would be produced using methods that assume simple random sampling.
Standard errors can be used as a measure of the precision expected from a particular sample.
Confidence intervals provide a way to make inferences about population statistics in a manner that reflects the sampling error associated with the statistic. Assuming a normal distribution and a 95 percent confidence interval, the population value of this statistic can be inferred to lie within the confidence interval in 95 out of 100 replications of the measurement on different samples drawn from the same population.
Confidentiality analyses for the United States were designed to provide reasonable assurance that public-use data files issued by the TALIS consortium would not allow identification of individual U.S. schools or teachers when compared against other public-use data databases. To limit disclosure risk for TALIS schools and teachers, analyses were conducted to identify potential risks, and based on this analysis school and teacher identification were masked by randomly swapping identifying data elements within the teacher and school files to add a measure of uncertainty. Swapping was designed and implemented in such a way that accurate estimates of means and variances for the whole population and reported subgroups could be maintained (Krenzke et al. 2006).
Comparisons made in the text of the TALIS Highlights have been tested for statistical significance. For example, in the commonly made comparison of TALIS averages to U.S. averages, tests of statistical significance were used to establish whether or not the observed differences from the U.S. average were statistically significant.
In almost all instances, the tests for significance used were standard t tests. These fell into two categories according to the nature of the comparison being made: comparisons of independent samples and comparisons of non-independent samples. In TALIS, education system groups are independent. We judge that a difference is “significant” if the probability associated with the t test is less than .05. If a test is significant this implies that difference in the observed means in the sample represents a real difference in the population.3 No adjustments were made for multiple comparisons.
In simple comparisons of independent averages, such as the average percentages of education system 1 with that of education system 2, the following formula was used to compute the t statistic:
where est1 and est2 are the estimates being compared (e.g., averages of education system 1 and education system 2) and se12 and se22 are the corresponding squared standard errors of these averages. The TALIS 2018 data are hierarchical and include school and teacher data from the participating schools. The standard errors for each education system take into account the clustered nature of the sampled data. These standard errors are not adjusted for correlations between groups since groups are independent.
The second type of comparison occurs when evaluating differences between non-independent groups within the education system. Because of the sampling design in which schools and teachers within schools are randomly sampled, the data within the education system from mutually exclusive sets of teachers or principals (for example, males and females) are not independent. For example, to determine whether the performance of females differs from that of males would require estimating the correlation between females’ and males’ scores. The BRR procedure previously mentioned was used to estimate the standard errors of differences between non-independent samples within the United States. Use of the BRR procedure implicitly accounts for the correlation between groups when calculating the standard errors.
To test comparisons between non-independent groups the following t statistic formula was used:
where estgrp1 and estgrp2 are the non-independent group estimates being compared and se(grp1-grp2) is the standard error of the difference calculated using BRR to account for the correlation between the estimates for the two non-independent groups.
3 A .05 probability implies that the t statistic is among the 5 percent most extreme values one would expect if there were no difference between the means. The decision rule is that when t statistics are this extreme, they are sampled from a population in which there is a difference between the means.
This section describes the success of participating education systems in meeting the international technical standards on data collection. Information is provided for all participating education systems on their coverage of the target population, exclusion rates, and response rates. For TALIS 2018, the principal and teacher data were adjudicated separately and are provided as separate, independent datasets with separate response rates.4
Table A-1 provides information on weighted school participation rates based on principals that responded to the principal questionnaire before and after school replacement and the number of participating schools after replacement for each participating education system. Table A-2 provides information on weighted school participation rates based on teachers that responded to the teacher questionnaire before and after school replacement and the number of participating schools and teachers after replacement for each participating education system.
Table A-3 provides information on the coverage of the target population and overall exclusion rates for each participating education system.
For the school data based on the participation of school principals, out of a sample of 220 schools, 214 were determined to be eligible and of those, 123 original schools participated for an initial unweighted response rate of 58 percent (63 percent weighted) before replacements were added. An additional 41 replacements schools participated for a total of 164 participating schools in the U.S. administration of TALIS. The unweighted response rate increased to 77 percent (also 78 percent weighted).
For school data based on the participation of teachers, 125 original schools participated for an initial unweighted response rate of 58 percent (60 percent weighted) before replacements were added. An additional 40 replacements schools participated for a total of 165 participating schools in the U.S. administration of TALIS. The unweighted response rate increased to 77 percent (77 percent weighted).
Within the schools that participated, up to 20 ISCED Level 2 teachers within each school were selected to participate in TALIS 2018. For the U.S., 90 percent (weighted) of sampled teachers participated in TALIS.
In addition to the international response rate standards described in the prior section, the U.S. sample had to meet the statistical standards of the National Center for Education Statistics (NCES) of the U.S. Department of Education. Since the U.S. TALIS weighted school response rate fell below 85 percent, NCES required that an investigation into the potential magnitude of nonresponse bias at the school level in the U.S. sample be conducted. Because the U.S. TALIS weighted teacher response rate was above 85 percent, a nonresponse bias analysis at the teacher level was not required.
Table A-1. Weighted participation rates and number of schools based on participating principals, by education system: 2018
Education system | Estimated size of school population | Weighted school participation before replacement | Weighted school participation after replacement | Number of participating schools after replacement |
---|---|---|---|---|
Alberta–Canada | 1,038 | 46.0 | 57.8 | 129 |
Australia | 2,680 | 47.1 | 75.1 | 230 |
Austria | 1,483 | 93.1 | 100.0 | 277 |
Belgium | 1,169 | 86.0 | 95.8 | 307 |
Belgium–Flemish | 721 | 81.4 | 94.0 | 188 |
Brazil | 52,187 | 88.0 | 95.4 | 184 |
Bulgaria | 1,730 | 96.5 | 100.0 | 200 |
Chile | 5,214 | 78.9 | 87.6 | 169 |
Chinese Taipei | 935 | 100.0 | 100.0 | 202 |
Colombia | 10,392 | 66.6 | 69.6 | 141 |
Croatia | 896 | 95.0 | 95.6 | 188 |
Cyprus | 99 | 88.9 | 88.9 | 88 |
Czech Republic | 2,606 | 99.0 | 99.0 | 218 |
Denmark | 1,457 | 51.5 | 71.4 | 140 |
England–United Kingdom | 3,990 | 70.1 | 81.8 | 157 |
Estonia | 389 | 88.3 | 100.0 | 195 |
Finland | 706 | 100.0 | 100.0 | 148 |
France | 6,770 | 97.6 | 98.0 | 195 |
Georgia | 2,151 | 91.7 | 91.7 | 177 |
Hungary | 2,640 | 90.9 | 93.3 | 182 |
Iceland | 136 | 74.3 | 74.3 | 101 |
Israel | 1,196 | 90.9 | 93.7 | 184 |
Italy | 5,622 | 92.4 | 98.6 | 190 |
Japan | 10,071 | 93.9 | 99.4 | 195 |
Kazakhstan | 6,302 | 100.0 | 100.0 | 331 |
Korea, Republic of | 3,134 | 68.1 | 77.8 | 150 |
Latvia | 653 | 73.4 | 90.8 | 136 |
Lithuania | 833 | 100.0 | 100.0 | 195 |
Malta | 58 | 93.1 | 93.1 | 54 |
Mexico | 16,327 | 90.6 | 97.0 | 193 |
Netherlands | 524 | 56.2 | 85.6 | 125 |
New Zealand | 1,732 | 71.7 | 92.0 | 189 |
Norway | 1,091 | 64.9 | 80.6 | 162 |
Portugal | 1,255 | 97.7 | 100.0 | 200 |
Romania | 4,658 | 100.0 | 100.0 | 199 |
Russian Federation | 31,948 | 99.1 | 100.0 | 230 |
Saudi Arabia | 6,119 | 96.2 | 96.2 | 192 |
Shanghai–China | 630 | 100.0 | 100.0 | 198 |
Singapore | 193 | 93.3 | 97.9 | 167 |
Slovak Republic | 1,593 | 83.6 | 90.4 | 180 |
Slovenia | 448 | 74.8 | 79.3 | 119 |
South Africa | 8,026 | 91.1 | 91.1 | 169 |
Spain | 6,861 | 98.1 | 98.5 | 396 |
Sweden | 1,739 | 83.5 | 88.6 | 171 |
Turkey | 16,100 | 98.9 | 98.9 | 196 |
United Arab Emirates | 521 | 91.4 | 91.4 | 476 |
United States | 65,095 | 63.1 | 77.6 | 164 |
Vietnam | 10,799 | 100.0 | 100.0 | 196 |
NOTE: Italics indicate non-OECD countries and education systems.
SOURCE: Organization for Economic Cooperation and Development (OECD), Teaching and Learning International Survey (TALIS), 2018.
Table A-2. Weighted participation rates and number of schools and teachers based on participating teachers, by education system: 2018
Education system | Estimated size of teacher population | Weighted school participation before replacement | Weighted school participation after replacement | Number of participating schools after replacement | Number of participating teachers | Teachers’ participation in participating schools (%) |
---|---|---|---|---|---|---|
Alberta–Canada | 9,991 | 48.9 | 60.3 | 122 | 1,077 | 83.0 |
Australia | 116,679 | 48.5 | 75.1 | 233 | 3,573 | 77.7 |
Austria | 45,869 | 85.7 | 88.6 | 246 | 4,255 | 84.4 |
Belgium | 34,442 | 86.0 | 95.1 | 302 | 5,257 | 86.9 |
Belgium–Flemish | 18,615 | 80.1 | 91.0 | 182 | 3,122 | 84.4 |
Brazil | 568,510 | 89.9 | 96.6 | 185 | 2,447 | 94.9 |
Buenos Aries–Argentina | 10,218 | 80.7 | 85.2 | 130 | 2,099 | 88.6 |
Bulgaria | 21,208 | 97.1 | 100.0 | 200 | 2,862 | 98.3 |
Chile | 55,969 | 82.6 | 91.5 | 179 | 1,963 | 94.3 |
Chinese Taipei | 53,208 | 98.9 | 98.9 | 200 | 3,835 | 97.2 |
Colombia | 164,225 | 73.1 | 77.1 | 154 | 2,398 | 93.4 |
Croatia | 15,762 | 95.4 | 96.2 | 188 | 3,358 | 87.0 |
Cyprus | 3,860 | 88.9 | 88.9 | 88 | 1,611 | 90.3 |
Czech Republic | 42,348 | 100.0 | 100.0 | 219 | 3,447 | 93.8 |
Denmark | 22,475 | 51.1 | 72.0 | 141 | 2,001 | 86.8 |
England–United Kingdom | 193,134 | 72.7 | 81.5 | 149 | 2,376 | 83.6 |
Estonia | 7,354 | 86.6 | 100.0 | 195 | 3,004 | 95.2 |
Finland | 18,938 | 100.0 | 100.0 | 148 | 2,851 | 96.2 |
France | 197,013 | 87.3 | 87.8 | 176 | 3,006 | 88.1 |
Georgia | 38,195 | 99.5 | 99.5 | 192 | 3,101 | 95.8 |
Hungary | 44,018 | 94.9 | 97.7 | 189 | 3,245 | 95.0 |
Iceland | 1,883 | 89.7 | 89.7 | 123 | 1,292 | 75.8 |
Israel | 32,603 | 84.9 | 86.4 | 172 | 2,627 | 84.9 |
Italy | 190,447 | 92.8 | 99.1 | 191 | 3,612 | 93.8 |
Japan | 230,558 | 92.5 | 99.5 | 196 | 3,555 | 99.0 |
Kazakhstan | 195,383 | 100.0 | 100.0 | 331 | 6,566 | 99.8 |
Korea, Republic of | 75,654 | 69.9 | 82.5 | 163 | 2,931 | 92.2 |
Latvia | 12,003 | 77.1 | 91.2 | 135 | 2,315 | 87.9 |
Lithuania | 19,848 | 100.0 | 100.0 | 195 | 3,759 | 97.4 |
Malta | 1,941 | 92.5 | 92.5 | 55 | 1,656 | 86.5 |
Mexico | 254,794 | 90.4 | 96.3 | 193 | 2,926 | 94.3 |
Netherlands | 66,672 | 56.7 | 79.5 | 116 | 1,884 | 80.9 |
New Zealand | 23,227 | 62.6 | 79.3 | 185 | 2,257 | 79.6 |
Norway | 21,828 | 77.4 | 92.6 | 185 | 4,154 | 83.2 |
Portugal | 39,703 | 97.9 | 100.0 | 200 | 3,676 | 92.7 |
Romania | 66,039 | 100.0 | 100.0 | 199 | 3,658 | 98.3 |
Russian Federation | 646,405 | 98.7 | 100.0 | 230 | 4,011 | 99.9 |
Saudi Arabia | 99,661 | 89.7 | 89.7 | 179 | 2,744 | 86.0 |
Shanghai–China | 38,902 | 100.0 | 100.0 | 198 | 3,976 | 99.5 |
Singapore | 11,544 | 96.9 | 100.0 | 169 | 3,280 | 99.2 |
Slovak Republic | 24,746 | 82.4 | 88.9 | 176 | 3,015 | 95.4 |
Slovenia | 7,422 | 82.2 | 88.0 | 132 | 2,094 | 91.5 |
South Africa | 92,127 | 92.1 | 92.4 | 170 | 2,046 | 89.7 |
Spain | 186,171 | 99.7 | 100.0 | 399 | 7,407 | 94.6 |
Sweden | 31,421 | 89.1 | 93.9 | 180 | 2,782 | 81.3 |
Turkey | 277,187 | 99.0 | 99.0 | 196 | 3,952 | 98.5 |
United Arab Emirates | 14,489 | 100.0 | 100.0 | 521 | 8,648 | 96.0 |
United States | 1,144,751 | 60.1 | 76.8 | 165 | 2,560 | 89.6 |
Vietnam | 295,033 | 100.0 | 100.0 | 196 | 3,825 | 96.3 |
NOTE: Italics indicate non-OECD countries and education systems.
SOURCE: Organization for Economic Cooperation and Development (OECD), Teaching and Learning International Survey (TALIS), 2018.
Table A-3. Coverage of ISCED Level 2 target population and survey population, by education system: 2018
Education system | Population and coverage: | Reasons for exclusions | Number of schools | Percentage of schools | Number of teachers | Percentage of teachers |
---|---|---|---|---|---|---|
Alberta—Canada | Target Population | 1,158 | 100 | — | — | |
Exclusive: | Band-operated schools. These schools operate on First Nations’ reserves and are the responsibility of the federal government rather than the responsibility of Alberta Education | 39 | 3.4 | — | — | |
Very small schools (fewer than three students in each of Grades 7 to 9) | 100 | 8.6 | — | — | ||
Survey Population: | 1,019 | 88 | — | — | ||
|
||||||
Australia | Target Population | 2,862 | 100 | — | — | |
Exclusive: | Small schools (fewer than four students) | 263 | 9.2 | — | — | |
Schools that are geographically remote | 32 | 1.1 | — | — | ||
Small schools that are also geographically remote | 31 | 1.1 | — | — | ||
Survey Population: | 2,536 | 88.6 | — | — | ||
|
||||||
Austria | Target Population | 1,496 | 100 | — | — | |
Exclusive: | Slovene school where the language of instruction is not German | 1 | 0.1 | — | — | |
Survey Population: | 1,495 | 99.9 | — | — | ||
|
||||||
Belgium | Target Population | 1,245 | 100 | — | — | |
Exclusive: | Very small schools (fewer than 20 students at ISCED level 2) | 10 | 0.8 | — | — | |
Special needs schools | 79 | 6.3 | — | — | ||
Survey Population: | 1,156 | 92.9 | — | — | ||
|
||||||
Belgium—Flemish | Target Population | 718 | 100 | — | — | |
Exclusive: | Very small schools (fewer than 20 students at ISCED level 2) | 2 | 0.3 | — | — | |
Survey Population: | 716 | 99.7 | — | — | ||
|
||||||
Brazil | Target Population | 58,303 | 100 | 870,737 | 100 | |
Exclusive: | Schools with fewer than six teachers. Because ISCED level 2 requires at least one teacher for each subject, most of these schools have only one class. The schools that fit this criterion are located in geographically remote areas | 4,957 | 8.5 | 18,896 | 2.2 | |
Public-federal schools | 38 | 0.1 | 1,724 | 0.2 | ||
Survey Population: | 53,308 | 91.4 | 850,117 | 97.6 | ||
|
||||||
Buenos Aires—Argentina | Target Population | 488 | 100 | — | — | |
Survey Population: | 488 | 100 | — | — | ||
|
||||||
Bulgaria | Target Population | 1,834 | 100 | 23,168 | 100 | |
Exclusive: | Schools for students with special education needs | 67 | 3.7 | 454 | 2.0 | |
Very small schools | 45 | 2.5 | 126 | 0.5 | ||
Survey Population: | 1,722 | 93.9 | 22,588 | 97.5 | ||
|
||||||
Chinese Taipei | Target Population | 939 | 100 | — | — | |
Schools with fewer than three teachers | 7 | 0.7 | — | — | ||
Survey Population: | 932 | 99.3 | — | — | ||
|
||||||
Chile | Target Population | 6,008 | 100 | 51,626 | 100 | |
Exclusions: | Schools with fewer than three teachers | 681 | 11.3 | 1433 | 2.8 | |
Schools that are geographically remote | 4 | 0.1 | 27 | 0.1 | ||
Survey Population: | 5,323 | 88.6 | 50,166 | 97.2 | ||
|
||||||
Colombia | Target Population | 13,009 | 100 | — | — | |
Exclusions: | Small schools (fewer than three teachers at ISCED level 2) | 322 | 2.5 | — | — | |
Survey Population: | 12,672 | 97.4 | — | — | ||
|
||||||
Croatia | Target Population | 860 | 100 | — | — | |
Exclusions: | National minority schools (Italian, Serbian schools) | 11 | 1.3 | — | — | |
Survey Population: | 849 | 98.7 | — | — | ||
|
||||||
Cyprus | Target Population | 102 | 100 | 4,426 | 100 | |
Exclusions: | School that is geographically remote | 1 | 1.0 | 13 | 0.3 | |
School where language of instruction is one other than Greek or English | 1 | 1.0 | 8 | 0.2 | ||
Very small school that has part-time teachers and no head teacher or assistant head teacher | 1 | 1.0 | 5 | 0.1 | ||
Survey Population: | 99 | 97.1 | 4,400 | 99.4 | ||
|
||||||
Czech Republic | Target Population | 2,645 | 100 | 39,690 | 100 | |
Exclusions: | Schools with a different language of instruction (Polish) | 10 | 0.4 | 116 | 0.3 | |
Dancing conservatoire – specific education programs | 5 | 0.2 | 122 | 0.3 | ||
Very small schools (fewer than three teachers at ISCED level 2) | 14 | 0.5 | 11 | 0.0 | ||
Survey Population: | 2,616 | 98.9 | 39,441 | 99.4 | ||
|
||||||
Denmark | Target Population | 1,721 | 100.0 | — | — | |
Exclusions: | Small schools (fewer than 40 students and generally fewer than 5 teachers) | 251 | 14.6 | — | — | |
Survey Population: | 1,470 | 85.4 | — | — | ||
|
||||||
England—United Kingdom | Target Population | 4,345 | 100 | — | — | |
Exclusions: | International schools | 13 | 0.3 | — | — | |
Very small schools | 57 | 1.3 | — | — | ||
Schools proposed for closure | 17 | 0.4 | — | — | ||
Survey Population: | 4,258 | 98.0 | — | — | ||
|
||||||
Estonia | Target Population | 404 | 100 | 8,622 | 100 | |
Exclusions: | International schools | 4 | 1.0 | 34 | 0.4 | |
Ballet school | 1 | 0.2 | 21 | 0.2 | ||
Survey Population: | 399 | 98.8 | 8,567 | 99.4 | ||
|
||||||
Finland | Target Population | 722 | 100 | — | — | |
Exclusions: | International//foreign/immersion schools, where all students are taught in languages other than Finnish or Swedish | 8 | 1.1 | — | — | |
Survey Population: | 714 | 98.9 | — | — | ||
|
||||||
France | Target Population | 7,203 | 100 | 215,940 | 100 | |
Exclusions: | Schools located in overseas French territories (TOM) | 79 | 1.1 | 1,342 | 0.6 | |
Schools located in La Réunion and Mayotte (Southern Hemisphere calendar) | 103 | 1.4 | 5,529 | 2.6 | ||
Private schools under different administration | 193 | 2.7 | — | — | ||
Survey Population: | 6,828 | 94.8 | 209,069 | 96.8 | ||
|
||||||
Georgia | Target Population | 2,265 | 100 | 42,757 | 100 | |
Exclusions: | Schools where the language of instruction is an excluded minority language | 13 | 0.6 | 249 | 0.6 | |
Very small schools | 2 | 0.1 | 6 | 0.0 | ||
Survey Population: | 2,250 | 99.3 | 42,502 | 99.4 | ||
|
||||||
Hungary | Target Population | 2,844 | 100 | 37,938 | 100 | |
Exclusions: | Very small schools (fewer than three teachers at ISCED level 2) | 85 | 3.0 | 126 | 0.3 | |
Survey Population: | 2,759 | 97.0 | 37,812 | 99.7 | ||
|
||||||
Iceland | Target Population | 142 | 100 | — | — | |
Survey Population: | 142 | 100 | — | — | ||
|
||||||
Israel | Target Population | 2,475 | 100 | — | — | |
Exclusions: | International/foreign schools where the language of instruction is a single minority language | 5 | 0.2 | — | — | |
Ultra-Orthodox Jewish schools where the language of instruction is Hebrew | 1,246 | 50.3 | — | — | ||
Survey Population: | 1,224 | 49.5 | — | — | ||
|
||||||
Italy | Target Population | 5,783 | 100 | 154,071 | 100 | |
Exclusions: | Schools with fewer than three teachers at ISCED level 2 | 63 | 1.1 | 90 | 0.1 | |
Survey Population: | 5,720 | 98.9 | 153,981 | 99.9 | ||
|
||||||
Japan | Target Population | 10,426 | 100 | 264,356 | 100 | |
Survey Population: | 10,426 | 100 | 264,356 | 100 | ||
|
||||||
Kazakhstan | Target Population | 6,424 | 100 | 208,254 | 100 | |
Exclusions: | Uzbek, Uighur and Tadjik schools. These regions together account for less than 5% of Kazakhstan’s population. The TALIS questionnaire will be administered in the Kazakh and Russian languages in these schools | 30 | 0.5 | 1,215 | 0.6 | |
Schools located in the Baikonur region (special permission is required to enter this territory | 5 | 0.1 | 306 | 0.1 | ||
Schools with fewer than three teachers at ISCED level 2 | 2 | 0.0 | 5 | 0.0 | ||
School where the language of instruction is English | 1 | 0.0 | 60 | 0.0 | ||
Survey Population: | 6,386 | 99.4 | 206,668 | 99.2 | ||
|
||||||
Korea, Republic of | Target Population | 3,252 | 100 | 69,820 | 100 | |
Exclusions: | Schools with fewer than three teachers at ISCED level 2 Schools that are geographically remote | 151 | 4.6 | 1,073 | 1.5 | |
Schools with no more than three teachers at ISCED level 2 and schools that are geographically remote | 4 | 0.1 | 7 | 0.0 | ||
Schools with alternative curricula, teacher licensure and school structure | 31 | 1.0 | 382 | 0.5 | ||
Survey Population: | 3,059 | 94.1 | 68,341 | 97.9 | ||
|
||||||
Latvia | Target Population | 695 | 100 | — | — | |
Exclusions: | International schools (e.g. for diplomats’ children) | 2 | 0.3 | — | — | |
Special regime (criminal) school | 1 | 0.1 | — | — | ||
Survey Population: | 692 | 99.6 | — | — | ||
|
||||||
Lithuania | Target Population | 960 | 100 | — | — | |
Exclusions: | International schools where English and French are the languages of instruction and the education system | 3 | 0.3 | — | — | |
Youth schools. These implement basic education programs but also have a particular focus on special education (emotionally or socially disadvantaged students) | 12 | 1.3 | — | — | ||
Small schools (fewer than ten students at ISCED level 2) | 19 | 2.0 | — | — | ||
Survey Population: | 926 | 96.5 | — | — | ||
|
||||||
Malta | Target Population | 63 | 100 | 3,271 | 100 | |
Exclusions: | Language schools | 2 | 3.2 | 16 | 0.5 | |
Survey Population: | 61 | 96.8 | 3,255 | 99.5 | ||
|
||||||
Mexico | Target Population | 16,763 | 100 | 328,649 | 100 | |
Exclusions: | Schools with fewer than three teachers | 41 | 0.2 | 95 | 0.0 | |
Survey Population: | 16,722 | 99.8 | 328,554 | 100.0 | ||
|
||||||
Netherlands | Target Population | 538 | 100 | — | — | |
Survey Population: | 538 | 100 | — | — | ||
|
||||||
New Zealand | Target Population | 1,696 | 100 | — | — | |
Exclusions: | National correspondence school | 1 | 0.1 | — | — | |
Survey Population: | 1,695 | 99.9 | — | — | ||
|
||||||
Norway | Target Population | 1,154 | 100 | 23,542 | 100 | |
Exclusions: | Very small schools (fewer than three teachers) | 68 | 5.9 | 168 | 0.7 | |
Sami schools | 5 | 0.4 | 48 | 0.2 | ||
International schools | 15 | 1.3 | 216 | 0.9 | ||
International schools where the language of instruction is French | 2 | 0.2 | 31 | 0.1 | ||
Survey Population: | 1,064 | 92.2 | 23,079 | 98.0 | ||
|
||||||
Portugal | Target Population | 1,273 | 100 | 36,912 | 100 | |
Exclusions: | Schools with non-Portuguese curricula | 16 | 1.3 | 299 | 0.8 | |
Survey Population: | 1,257 | 98.7 | 36,613 | 99.2 | ||
|
||||||
Romania | Target Population | 4,776 | 100 | — | — | |
Exclusions: | Very small schools (fewer than 25 students at ISCED level 2) | 88 | 1.8 | — | — | |
Survey Population: | 4,688 | 98.2 | — | — | ||
|
||||||
Russian Federation | Target Population | 41,893 | 100 | 760,196 | 100 | |
Exclusions: | Crimean schools | 621 | 1.5 | 12,462 | 1.6 | |
Moscow schools | 733 | 1.7 | 41,541 | 5.5 | ||
Survey Population: | 40,539 | 96.8 | 706,193 | 92.9 | ||
|
||||||
Saudi Arabia | Target Population | 8,105 | 100 | 120,109 | 100 | |
Exclusions: | Schools in the Najran and Jazan regions (near the borders with Yemen where there is war) | 767 | 9.5 | 9,028 | 7.5 | |
Private schools. These schools are not being included in TALIS at this stage because their system of education differs from that of the public system | 951 | 11.7 | 11,075 | 9.2 | ||
Schools with fewer than four teachers | 121 | 1.5 | 240 | 0.2 | ||
Survey Population: | 6,266 | 77.3 | 99,766 | 83.1 | ||
|
||||||
Shanghai—China | Target Population | 662 | 100 | 41,705 | 100 | |
Exclusions: | Special schools for students with behavioral problems, including delinquency | 12 | 1.8 | 340 | 0.8 | |
Survey Population: | 650 | 98.2 | 41,365 | 99.2 | ||
|
||||||
Singapore | Target Population | 197 | 100 | 12,285 | 100 | |
Exclusions: | Schools where the language of instruction is not English and where teachers would have difficulty responding to the TALIS survey, which is in English | 4 | 2.0 | 200 | 1.6 | |
Survey Population: | 193 | 98.0 | 12,085 | 98.4 | ||
|
||||||
Slovak Republic | Target Population | 1,591 | 100 | 24,841 | 100 | |
Exclusions: | Schools with fewer than three teachers at ISCED level 2 | 10 | 0.6 | 20 | 0.1 | |
Survey Population: | 1,581 | 99.4 | 24,821 | 99.9 | ||
|
||||||
Slovenia | Target Population | 451 | 100 | 9,090 | 100 | |
Exclusions: | Italian basic schools in an ethnically mixed area where the language of instruction is Italian and where the teachers are therefore a separate group of Slovenian teachers | 3 | 0.7 | 42 | 0.5 | |
Survey Population: | 448 | 99.3 | 9,048 | 99.5 | ||
|
||||||
Spain | Target Population | 6,954 | 100 | 200,193 | 100 | |
Exclusions: | Small schools (fewer than three teachers) | 43 | 0.6 | 59 | 0.0 | |
Schools that are geographically remote | 2 | 0.0 | 42 | 0.0 | ||
Survey Population: | 6,909 | 99.4 | 200,092 | 100.0 | ||
|
||||||
Sweden | Target Population | 1,768 | 100 | — | — | |
Exclusions: | Small schools (fewer than six students at ISCED level 2) | 49 | 2.8 | — | — | |
International schools not following the Swedish curriculum | 11 | 0.6 | — | — | ||
Survey Population: | 1,708 | 96.6 | — | — | ||
|
||||||
Turkey | Target Population | 16,228 | 100 | 310,932 | 100 | |
Survey Population: | 16,228 | 100 | 310,932 | 100 | ||
|
||||||
United Arab Emirates | Target Population | 577 | 100 | 17,541 | 100 | |
Exclusions: | Private schools | 14 | 2.4 | 350 | 2.0 | |
Survey Population: | 563 | 97.6 | 17,191 | 98.0 | ||
|
||||||
United States | Target Population | 63,795 | 100 | — | — | |
Exclusions: | Schools in detention, hospital and treatment centers | 569 | 0.9 | — | — | |
Survey Population: | 63,226 | 99.1 | — | — | ||
|
||||||
Vietnam | Target Population | 10,843 | 100 | 303,171 | 100 | |
Exclusions: | Non-Vietnamese international schools | 22 | 0.2 | 153 | 0.1 | |
Survey Population: | 10,821 | 99.8 | 303,018 | 99.9 | ||
|
— Not available
NOTE: Detail may not sum to totals because of rounding. Italics indicate non-OECD countries and education systems.
SOURCE: Organization for Economic Cooperation and Development (OECD), Teaching and Learning International Survey (TALIS), 2018
4 Principal/school data were adjudicated on their own in the 2018 survey, an occurrence that resulted in the notion of a “participating school for principal/school data” being introduced. A school was considered “participating” if its principal returned his or her questionnaire with at least one valid response. For the teacher data, the minimum of 50% teacher participation remained the criterion for determining whether a school was “participating” or not. Consequently, and in contrast to TALIS 2008 and 2013, a school record remained on the school file if the principal responded to the questionnaire, even if fewer than 50% of the teachers in the school participated in the survey.
In addition to the international response rate standards described in the prior section, the U.S. sample had to meet the statistical standards of the National Center for Education Statistics (NCES) of the U.S. Department of Education. Since the U.S. TALIS weighted school response rate fell below 85 percent, NCES required that an investigation into the potential magnitude of nonresponse bias at the school level in the U.S. sample be conducted. Because the U.S. TALIS weighted teacher response rate was above 85 percent, a nonresponse bias analysis at the teacher level was not required.
Of the 220 original sampled schools in the national sample, 214 were eligible (six schools did not have any grade-eligible students or had closed), and 125 agreed to participate. The weighted school response rate before replacement was 60 percent requiring the United States to conduct a nonresponse bias analysis to evaluate the quality of the final United States sample.
A bias analysis was conducted to address potential problems in the data owing to school nonresponse. The general approach taken involves an analysis in three parts as described below.
To compare TALIS participating schools to the total eligible sample of schools, it was necessary to match the sample of schools to the sample frame to identify as many characteristics as possible that might provide information about the presence of nonresponse bias. Frame characteristics were taken from the 2015–16 Common Core of Data for public schools and from the 2015–16 Private School Universe Survey for private schools. The available school characteristics included school control (public or private), locale (central city, suburb, town, rural), census region, poverty (high or low poverty for public schools based on whether 50 percent or more of the students are eligible for participation in the national free and reduced-price lunch (FRPL) program), grade structure (schools with grades 6–8, grades 9–12, other), School size (small, medium, and large), number of grade-eligible students, total number of students, and percentage of students in various racial/ethnic groups (White, non-Hispanic; Black, non-Hispanic; Hispanic; Asian; American Indian or Alaska Native; Hawaiian/Pacific Islander; and two or more races). The percentage of students eligible for free or reduced-price lunch was available for public schools only. The full text of the nonresponse bias analysis conducted for TALIS 2018 will be included in the technical report released with the U.S. national dataset (forthcoming).
For original sample schools, participating schools had a lower mean total school enrollment than the eligible sample (812.6 versus 908.6, respectively; and a lower mean grade-eligible enrollment than the eligible sample (374.5 versus 424.8, respectively). Additionally, the absolute value of the relative bias for private schools, schools in rural areas, schools in each of the four census regions, small sized and large sized schools, Asian, American Indian or Alaska Native, and Hawaiian/Pacific Islander is greater than 10 percent, which indicates potential bias even though no statistically significant relationship was detected. Although each of these findings indicates some potential for nonresponse bias, when all of these factors were considered simultaneously in a regression analysis, only town was a significant predictor of school participation. The percentage of students eligible for free or reduced-price lunch was not included in the logistic regression analysis as public and private schools were modeled together using only the variables available for all schools.
For the United States final sample schools (with substitutes), schools in the Northeast were underrepresented among participating schools relative to eligible schools (15.8 percent versus 19.6 percent, respectively), while schools in the South were overrepresented among participating schools (47.6 percent versus 43.1 percent, respectively). Participating schools had a lower mean total school enrollment than the eligible sample (767.3 versus 857.6, respectively) and a lower mean grade-eligible enrollment than the eligible sample (373.8 versus 405.1, respectively). Participating schools had a lower mean percentage of Asian students than the eligible sample (3.7 percent versus 4.3 percent, respectively). Additionally, the absolute value of the relative bias for schools in towns and rural areas and large sized schools are greater than 10 percent, which indicates potential bias even though no statistically significant relationships were detected. When all factors were considered simultaneously in a logistic regression analysis (again with free or reduced-price lunch eligibility omitted), no variables were statistically significant predictors of participation. The model among public schools only showed that South region, high poverty, interaction between the high-poverty indicator and free or reduced-price lunch eligibility, and summed race/ethnicity were significant predictors of school participation.
In the final sample schools with substitutes when school nonresponse adjusted weights were used for the participating schools, schools in the Northeast were underrepresented among participating schools relative to eligible schools (15.6 percent versus 19.6 percent, respectively), while schools in the South were overrepresented among participating schools (47.7 percent versus 43.1 percent, respectively). Participating schools had a lower mean total school enrollment than the eligible sample (785.9 versus 857.6, respectively) and a lower mean grade-eligible enrollment than the eligible sample (375.6 versus 405.1, respectively). Participating schools had a lower mean percentage of Asian students than the eligible sample (3.7 percent versus 4.3 percent, respectively). Additionally, the absolute value of the relative bias for schools in towns and rural areas and large sized schools is again greater than 10 percent, which indicates potential bias even though no statistically significant relationship was detected. The multivariate regression analysis cannot be conducted after the school nonresponse adjustments are applied to the weights. The concept of nonresponse adjusted weights does not apply to the nonresponding units, and, thus, we cannot conduct an analysis that compares respondents with nonrespondents using nonresponse adjusted weights.
In summary, the investigation into nonresponse bias at the school level in the U.S. TALIS 2018 provides evidence that there is some potential for nonresponse bias in the TALIS participating original sample based on the characteristics studied. It also suggests that there is no evidence that the use of substitute schools reduced the potential for bias. Moreover, after the application of school nonresponse adjustments, there is some evidence of resulting potential bias in the available frame variables in the final sample with the largest relative bias in census region.
1 The size-adjusted weight modifies the school base weight so that each sampled school is weighted in proportion to the number of teachers that it represents in the population. For considering potential nonresponse bias in TALIS, the estimated number of teachers of a school is a better measure of the relative size of a school than the estimated grade-eligible enrollment of a school, described in an earlier section, as TALIS is a survey of the population of teachers.
Ainley, J. and R. Carstens. (2018). Teaching and Learning International Survey (TALIS) 2018 Conceptual Framework, OECD Education Working Papers, No. 187, OECD Publishing, Paris. https://doi.org/10.1787/799337c2-en .
Judkins, D. R. (1990). Fay’s Method for Variance Estimation. Journal of Official Statistics 6(3): 223-239.
Kastberg, D., Cummings, L., Lemanski, N., Ferraro, D., Perkins, R.C., and Erberber, E. (forthcoming). U.S. Technical Report and User Guide for the 2018 Teaching and Learning International Survey (TALIS). U.S. Department of Education. Washington, DC: National Center for Education Statistics.
Krenzke, T., Roey, S., Dohrmann, S., Mohadjer, L., Haung, W., Kaufman, S., and Seastrom, M. (2006). Tactics for Reducing the Risk of Disclosure Using the NCES DataSwap Software. Proceedings of the Survey Research Methods Section 2006. Alexandria, VA: American Statistical Association.
Organization for Economic Cooperation and Development (OECD). (2019). TALIS 2018 Technical Report. Paris: Author.