Appendix A.5. Organization for Economic Cooperation and Development

The Organization for Economic Cooperation and Development (OECD) publishes analyses of national policies and survey data in education, training, and economics in OECD and partner countries. Newer studies include student survey data on financial literacy and on digital literacy.

Education at a Glance

To highlight current education issues and create a set of comparative education indicators that represent key features of education systems, OECD initiated the Indicators of Education Systems (INES) project and charged the Centre for Educational Research and Innovation (CERI) with developing the cross-national indicators for it. The development of these indicators involved representatives of the OECD countries and the OECD Secretariat. Improvements in data quality and comparability among OECD countries have resulted from the country-to-country interaction sponsored through the INES project. The most recent publication in this series is Education at a Glance 2020: OECD Indicators.

Education at a Glance 2020 features data on the 37 OECD countries (Australia, Austria, Belgium, Canada, Chile, Colombia, the Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Japan, the Republic of Korea, Latvia, Lithuania, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, the United Kingdom, and the United States) and a number of partner countries, including Argentina, Brazil, China, Costa Rica, India, Indonesia, the Russian Federation, Saudi Arabia, and South Africa.

The OECD Handbook for Internationally Comparative Education Statistics: Concepts, Standards, Definitions and Classifications provides countries with specific guidance on how to prepare information for OECD education surveys; facilitates countries’ understanding of OECD indicators and their use in policy analysis; and provides a reference for collecting and assimilating educational data. Chapter 6 of the OECD Handbook for Internationally Comparative Education Statistics contains a discussion of data quality issues. Users should examine footnotes carefully to recognize some of the data limitations.

Further information on international education statistics may be obtained from

Andreas Schleicher
Director for the Directorate of Education and Skills
  and Special Advisor on Education Policy
  to the OECD’s Secretary General
OECD Directorate for Education and Skills
2, rue André Pascal
75775 Paris Cedex 16
France
andreas.schleicher@oecd.org
https://www.oecd.org/

Top

Online Education Database (OECD.Stat)

The statistical online platform of the OECD, OECD.Stat, allows users to access OECD’s databases for OECD member countries and selected nonmember economies. A user can build tables using selected variables and customizable table layouts, extract and download data, and view metadata on methodology and sources.

Data for educational attainment in this report are pulled directly from OECD.Stat. (Information on these data can be found in chapter A, indicator A1, of annex 3 in Education at a Glance 2020 and accessed at https://www.oecd-ilibrary.org/education/education-at-a-glance-2020_442eac2f-en .) However, to support statistical testing for NCES publications, standard errors for some countries had to be estimated and therefore may not be included on OECD.Stat. Standard errors for 2018 and 2019 for Japan, the Republic of Korea, and Turkey; for 2018 for Poland; and for 2019 for Slovenia were estimated by NCES using a simple random sample assumption. These standard errors are likely to be lower than standard errors that take into account complex sample designs. Lastly, NCES estimated the standard errors for the OECD average using the sum of squares technique.

OECD.Stat can be accessed at https://stats.oecd.org/. A user’s guide for OECD.Stat can be accessed at https://stats.oecd.org/Content/themes/OECD/static/help/WBOS%20User%20Guide%20(EN).pdf.

Top

Program for International Student Assessment

The Program for International Student Assessment (PISA) is a system of international assessments organized by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries, that focuses on 15-year-olds’ capabilities in reading literacy, mathematics literacy, and science literacy. PISA also includes measures of general, or cross-curricular, competencies such as learning strategies. PISA emphasizes functional skills that students have acquired as they near the end of compulsory schooling.

PISA is a 2-hour exam. Assessment items include a combination of multiple-choice questions and open-ended questions that require students to develop their own response. PISA scores are reported on a scale that ranges from 0 to 1,000, with the OECD mean set at 500 and a standard deviation set at 100. In each education system, the assessment is translated into the primary language of instruction; in the United States, all materials are written in English.

Forty-three education systems participated in the 2000 PISA; 41 education systems participated in 2003; 57 (30 OECD member countries and 27 nonmember countries or education systems) participated in 2006; and 65 (34 OECD member countries and 31 nonmember countries or education systems) participated in 2009. (An additional nine education systems administered the 2009 PISA in 2010.) In PISA 2012, 65 education systems (34 OECD member countries and 31 nonmember countries or education systems), as well as the states of Connecticut, Florida, and Massachusetts, participated. In the 2015 PISA, 70 education systems (35 OECD member countries and 35 nonmember countries or education systems), as well as the states of Massachusetts and North Carolina and the territory of Puerto Rico, participated. In PISA 2018, 79 education systems (37 OECD member countries and 42 nonmember countries or education systems) participated.

To implement PISA, each of the participating education systems scientifically draws a nationally representative sample of 15-year-olds, regardless of grade level. In the 2018 PISA, there were 162 participating schools and 4,811 participating students. The overall weighted school response rate was 76 percent, and the overall weighted student response rate was 85 percent.

The intent of PISA reporting is to provide an overall description of performance in reading literacy, mathematics literacy, and science literacy every 3 years, and to provide a more detailed look at each domain in the years when it is the major focus. These cycles will allow education systems to compare changes in trends for each of the three subject areas over time. In the first cycle, PISA 2000, reading literacy was the major focus, occupying roughly two-thirds of assessment time. For 2003, PISA focused on mathematics literacy as well as the ability of students to solve problems in real-life settings. In 2006, PISA focused on science literacy; in 2009, it focused on reading literacy again; and in 2012, it focused on mathematics literacy. PISA 2015 focused on science, as it did in 2006. PISA 2018 focused on reading, as it did in 2009; it also offered an optional assessment of financial literacy, administered by the United States.

Further information on PISA may be obtained from

Samantha Burg
International Assessment Branch
Assessments Division
National Center for Education Statistics
550 12th Street SW
Washington, DC 20202
samantha.burg@ed.gov
https://nces.ed.gov/surveys/pisa/

Top

Program for the International Assessment of Adult Competencies

The Program for the International Assessment of Adult Competencies (PIAAC) is a cyclical, large-scale study that aims to assess and compare the broad range of basic skills and competencies of adults around the world. Developed under the auspices of the Organization for Economic Cooperation and Development (OECD), it is the most comprehensive international survey of adult skills ever undertaken. Adults were surveyed in 24 participating countries in 2012, 9 participating countries in 2014, and 5 participating countries in 2017.

PIAAC focuses on what are deemed basic cognitive and workplace skills necessary to adults’ successful participation in 21st-century society and in the global economy. Skills assessed include literacy, numeracy, problem solving in technology-rich environments, and basic reading skills. PIAAC measures the relationships between these skills and other characteristics such as individuals’ educational background, workplace experiences, and occupational attainment. PIAAC was administered on laptop computers or in paper-and-pencil format. In the United States, the background questionnaire was administered in both English and Spanish, and the cognitive assessment was administered only in English.

Cycle 1 of PIAAC consisted of three rounds of data collection: The first round (also referred to as the main study) was conducted in 2012; the second round (also referred to as the national supplement to the main study) was conducted in 2014; and the third round was conducted in 2017. In the first round (2012), the assessment for the United States included a nationally representative probability sample of households. This household sample was selected on the basis of a four-stage, stratified area sample: (1) primary sampling units (PSUs) consisting of counties or groups of contiguous counties; (2) secondary sampling units (referred to as segments) consisting of area blocks; (3) housing units containing households; and (4) eligible persons within households. Person-level data were collected through a screener, a background questionnaire, and the assessment.

Based on the screener data, 6,100 U.S. respondents ages 16 to 65 were selected to complete the 2012 background questionnaire and the assessment; 4,898 actually completed the background questionnaire. Of the 1,202 respondents who did not complete the background questionnaire, 112 were unable to do so because of a literacy-related barrier—either the inability to communicate in English or Spanish or a mental disability. Twenty others were unable to complete the questionnaire due to technical problems. The final response rate for the background questionnaire, which included respondents who completed it and respondents who were unable to complete it because of a language problem or mental disability, was 82.2 percent weighted. The overall person-weighted response rate for the household sample—the product of the component response rates—was 70.3 percent.

The second round of data collection, administered in 2014 to an additional sample of 3,660 U.S. adults, was conducted to (i) enhance the main study’s dataset by oversampling young adults (those ages 16–34) and unemployed adults, and (ii) expand the sample to include older adults (ages 66–74). The results of the U.S. PIAAC second round of data collection were combined, by design, with the U.S. PIAAC first round of data collection into one data point for 2012/14. The combined PIAAC 2012/2014 weighted response rate for the background questionnaire was 80.9 percent; the combined PIAAC 2012/2014 weighted overall response rate was 67.8 percent.

The third round of U.S. data collection (2017) was conducted with a nationally representative household sample of 3,660 adults between the ages of 16 and 74. The results from the 2017 data collection provide a second point in time for comparisons to the 2012/2014 results. The PIAAC 2017 weighted response rate for the background questionnaire was 76.3 percent; the PIAAC 2017 weighted overall response rate was 56.0 percent.

Key to PIAAC’s value is its collaborative and international nature. In the United States, NCES has consulted extensively with the Department of Labor in the development of the survey, and staff from both agencies are co-representatives of the United States in PIAAC’s international governing body. Internationally, PIAAC has been developed through the collaboration of OECD staff and participating countries’ representatives from their ministries or departments of education and labor. Through this cooperative effort, all participating countries follow the quality assurance guidelines set by the OECD consortium and closely follow all agreed-upon standards set for survey design, assessment implementation, and reporting of results.

Further information on PIAAC may be obtained from

Holly Xie
International Assessment Branch
Assessments Division
National Center for Education Statistics
550 12th Street SW
Washington, DC 20202
holly.xie@ed.gov
https://nces.ed.gov/surveys/piaac/
https://www.oecd.org/skills/piaac/

Top

Teaching and Learning International Survey (TALIS)

The Teaching and Learning International Survey (TALIS) is an international large-scale survey of the teachers, teaching, and the learning environments in schools conducted in 2008, 2013, and 2018 by the Organization for Economic Cooperation and Development (OECD). Data from the survey are based on questionnaire responses from nationally representative samples of teachers and their principals in participating countries and education systems.

The main objective of TALIS is to provide accurate and relevant international indicators on teachers and teaching, with the goal of helping countries review current conditions and develop informed education policy. The survey’s core target population is International Standard Classification of Education (ISCED) level 2 (lower secondary) teachers and school principals. ISCED level 2 corresponds to grades 7, 8, and 9 in the United States.

The sample design for TALIS 2018 was a stratified systematic sample, with the school sampling probability proportional to the estimated number of ISCED 2 teachers within each school. Samples were drawn using a two-stage sampling process. In the first stage, a sample of schools was drawn; in the second stage, a sample of teachers within each selected school was drawn.

A minimum sample size of 4,000 teachers from a minimum of 200 schools was required for each participating education system. Replacement schools were identified at the same time the TALIS sample was selected by designating the two neighboring schools in the sampling frame as replacement schools. Within schools, a sample of 20 teachers was to be selected in an equal probability sample unless fewer than 20 teachers were available (in which case all teachers were selected).

Each education system collected its own data following international guidelines and specifications. The technical standards required that eligible teachers were those teaching at least one ISCED Level 2 class, regardless of subject matter. School principals or head administrators of each sampled school were also asked to participate. School principal and teacher data were collected independently so that teacher eligibility was not dependent on principal participation (or vice versa).

The response-rate target was at least 75 percent of schools and at least 75 percent of teachers across the participating schools in each education system. A minimum participation rate of 50 percent of schools from the original school sample and 75 percent of schools after replacement was required in order for an education system’s data to be included in the main international comparisons. Education systems were allowed to use replacement schools (selected during the sampling process) to increase the response rate as long as the 50 percent benchmark before replacement had been reached.

The data collected by each participating education system was adjudicated to ensure that it met the TALIS technical standards for data collection. The principal and teacher data were adjudicated separately. For school-level data, adjudication depended only on school data (the principal participation); for teacher-level data, adjudication depended only on teacher data (50 percent of teachers in the school had to participate).

The United States first participated in TALIS in 2013, along with 37 other education systems. The most recent round of data collection was in 2018, with 49 education systems participating. U.S. results for the 2018 administration of TALIS are available at https://nces.ed.gov/surveys/talis/talis2018/, and full results from all three rounds of TALIS are available at https://www.oecd.org/education/talis/.

Further information on TALIS may be obtained from

Mary Coleman
International Assessment Branch
Assessments Division
National Center for Education Statistics
550 12th Street SW
Washington, DC 20202
mary.coleman@ed.gov
https://nces.ed.gov/surveys/talis/