Indicator 22: Problem Solving in Technology-Rich Environments Among 16- to 19-Year-Olds: National and International Comparisons

Overall, 4 percent of U.S. 16- to 19-year-olds performed at the highest proficiency level (level 3) on the Program for the International Assessment of Adult Competencies (PIAAC) problem solving in technology-rich environments scale, 31 percent performed at level 2, 47 percent performed at level 1, and 18 percent performed at the lowest proficiency level (below level 1). The percentage of U.S. 16- to 19-year-olds performing at the lowest proficiency level was higher than the OECD average (12 percent).

The Program for the International Assessment of Adult Competencies (PIAAC) is a large-scale survey that assesses and compares adult skills in literacy, reading components, numeracy, and problem solving in technology-rich environments around the world. The survey is administered by answering questions via computer, although the survey can also be implemented via paper-and-pencil. PIAAC is developed and organized by the Organization for Economic Cooperation and Development (OECD).

Problem solving and other cognitive skills in technology-rich environments are increasingly important in the economy and society of the 21st century. Problem solving in technology-rich environments is defined in PIAAC as using digital technology, communication tools, and networks to acquire and evaluate information, communicate with others, and perform practical tasks. PIAAC survey items in this domain present tasks of varying difficulty in simulated software applications using commands and functions commonly found in email, web pages, and spreadsheets.

Using PIAAC data from 2012, 2014, and 2015, this indicator describes the performance of U.S. 16- to 19-year-olds on the problem solving in technology-rich environments scale and differences in scores with respect to individual and family characteristics. In addition, this indicator compares the performance of U.S. 16- to 19-year-olds with their peers from selected OECD¹ and non-OECD countries and subnational educational systems. Although the indicator does not examine access to digital learning resources outside of the classroom, it does provide a general snapshot of 16- to 19-year-olds' capability of using digital learning resources by examining their performance in problem solving in technology-rich environments.

PIAAC reports four levels for the problem solving in technology-rich environments scale: below level 1, level 1, level 2, and level 3. Tasks at a higher level require using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher-level forms of reasoning.² This indicator primarily focuses on discussions about the lowest (below level 1) and highest (level 3) proficiency levels. However, most analyses are limited to the lowest proficiency level due to the small percentage of 16- to 19-year-olds performing at the highest proficiency level.

Caution should be used when comparing performance on the PIAAC performance of problem solving in technology-rich environments scale across countries. The percentage of respondents assessed in this domain varied widely from country to country, because items on problem solving in technology-rich environments were administered only on computers. Some respondents elected not to take a computer-based assessment or were unable to do so because they had limited or no familiarity with computers. For these reasons, 7 percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 percent (Belgium and Slovenia) to 34 percent (Turkey) across OECD countries. In the United States and across the OECD, a higher percentage of male than female 16- to 19-year-olds were not assessed in the scale. A higher percentage of U.S. 16- to 19-year-olds with neither parent attaining a high school degree were not assessed in the scale compared to those who had at least one parent with a college degree. Across the OECD, a higher percentage of 16- to 19-year-olds with neither parent attaining a high school degree were not assessed in the scale compared to those who had at least one parent with a high school or college degree.

---

Figure 22.1. Percentage distribution of 16- to 19-year-olds assessed in the problem solving in technology-rich environments domain for the United States and the OECD average, by sex and proficiency level: 2012, 2014, and 2015

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent.
¹ Refers to the mean of the data values for all reporting Organization for Economic Cooperation and Development (OECD) countries and subnational education systems, to which each country or subnational education system reporting data contributes equally.
NOTE: Program for International Assessment of Adult Competencies (PIAAC) results for the United States are based on combined data from 2012 and 2014. Results for the OECD were mostly collected in 2012, but a second round of data collection was completed in 2015 for countries that did not participate in 2012. Detail may not sum to totals because of rounding. The proficiency levels correspond to the following score ranges on a scale of 0 to 500: below level 1 (0–240.9), level 1 (241.0–290.9), level 2 (291.0–340.9), and level 3 (341.0–500.0). Tasks at a higher level are more demanding in terms of requirements such as using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher level forms of reasoning. For detailed descriptions of each proficiency level, as well as specific examples of tasks at each level, see appendix B of the report Skills of U.S. Unemployed, Young, and Older Adults in Sharper Focus (NCES 2016-039rev), available at http://nces.ed.gov/pubs2016/2016039rev.pdf. Seven percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 to 34 percent across OECD countries.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for the International Assessment of Adult Competencies (PIAAC), U.S. PIAAC 2012/2014; Organization for Economic Cooperation and Development, PIAAC 2012 and 2015. See Digest of Education Statistics 2016, table 604.50.

---

Overall, 18 percent of U.S. 16- to 19-year-olds performed at the lowest proficiency level (below level 1) on the problem solving in technology-rich environments scale, 47 percent performed at level 1, 31 percent performed at level 2, and 4 percent performed at the highest proficiency level (level 3). The percentage of U.S. 16- to 19-year-olds performing at the lowest proficiency level was larger than the OECD average³ (18 vs. 12 percent). In contrast, the percentage of U.S. 16- to 19-year-olds scoring at level 2 was smaller than the OECD average (31 vs. 41 percent). There were no measurable differences between the United States and the OECD average percentages of 16- to 19-year-olds performing at either level 1 or the highest proficiency level (level 3).

No measurable gaps were observed between the percentages of U.S. male and female 16- to 19-year-olds at each proficiency level on the problem solving in technology-rich environments scale. This pattern was also observed for the OECD average percentages at each level. However, the percentage of U.S. males performing at the lowest proficiency level was larger than the OECD average (19 vs. 12 percent).

---

Figure 22.2. Percentage distribution of U.S. 16- to 19-year-olds assessed in the problem solving in technology-rich environments domain, by race/ethnicity and proficiency level: 2012 and 2014

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent.
‡ Reporting standards not met. Either there are too few cases for a reliable estimate or the coefficient of variation (CV) is 50 percent or greater.
¹ The percentage of Asian/Pacific Islander 16- to 19-year-olds performing at below level 1 on the scale did not meet the reporting standards because the coefficient of variation (CV) is 50 percent or greater.
² "Other" includes persons of all other races and Two or more races.
NOTE: Program for International Assessment of Adult Competencies (PIAAC) results for the United States are based on combined data from 2012 and 2014. Results for the OECD were mostly collected in 2012, but a second round of data collection was completed in 2015 for countries that did not participate in 2012. Detail may not sum to totals because of rounding. The proficiency levels correspond to the following score ranges on a scale of 0 to 500: below level 1 (0–240.9), level 1 (241.0–290.9), level 2 (291.0–340.9), and level 3 (341.0–500.0). Tasks at a higher level are more demanding in terms of requirements such as using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher level forms of reasoning. For detailed descriptions of each proficiency level, as well as specific examples of tasks at each level, see appendix B of the report Skills of U.S. Unemployed, Young, and Older Adults in Sharper Focus (NCES 2016-039rev), available at http://nces.ed.gov/pubs2016/2016039rev.pdf. Seven percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 to 34 percent across OECD countries.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for the International Assessment of Adult Competencies (PIAAC), U.S. PIAAC 2012/2014. See Digest of Education Statistics 2016, table 604.50.

---

The percentages of U.S. 16- to 19-year-olds who performed at the lowest proficiency level on the problem solving in technology-rich environments scale varied across racial/ethnic groups. Larger percentages of Black (32 percent) and Hispanic (27 percent) 16- to 19-year-olds performed at the lowest proficiency level compared with White (12 percent) 16- to 19-year-olds.

---

Figure 22.3. Percentage distribution of 16- to 19-year-olds assessed in the problem solving in technology-rich environments domain for the United States and the OECD average, by highest level of parental education and proficiency level: 2012, 2014, and 2015

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent.
‡ Reporting standards not met. Either there are too few cases for a reliable estimate or the coefficient of variation (CV) is 50 percent or greater.
¹ Refers to the mean of the data values for all reporting Organization for Economic Cooperation and Development (OECD) countries and subnational education systems, to which each country or subnational education system reporting data contributes equally.
NOTE: Program for International Assessment of Adult Competencies (PIAAC) results for the United States are based on combined data from 2012 and 2014. Results for the OECD were mostly collected in 2012, but a second round of data collection was completed in 2015 for countries that did not participate in 2012. Detail may not sum to totals because of rounding. The proficiency levels correspond to the following score ranges on a scale of 0 to 500: below level 1 (0–240.9), level 1 (241.0–290.9), level 2 (291.0–340.9), and level 3 (341.0–500.0). Tasks at a higher level are more demanding in terms of requirements such as using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher level forms of reasoning. For detailed descriptions of each proficiency level, as well as specific examples of tasks at each level, see appendix B of the report Skills of U.S. Unemployed, Young, and Older Adults in Sharper Focus (NCES 2016-039rev), available at http://nces.ed.gov/pubs2016/2016039rev.pdf. Seven percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 to 34 percent across OECD countries.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for the International Assessment of Adult Competencies (PIAAC), U.S. PIAAC 2012/2014; Organization for Economic Cooperation and Development, PIAAC 2012 and 2015. See Digest of Education Statistics 2016, table 604.50.

---

The performance of 16- to 19-year-olds on the problem solving in technology-rich environments scale varied by parents' highest educational level, defined as the highest level of education attained by the most educated parent in the household. In the United States, the percentage of 16- to 19-year-olds at the lowest proficiency level was larger for those with at least one parent who had attained a high school degree (24 percent) than for those with at least one parent who had attained a college degree (13 percent). For the OECD average, the percentage of 16- to 19-year-olds performing at the lowest proficiency level was largest for those who had neither parent attaining a high school degree (27 percent) and smallest for those who had at least one parent attaining a college degree (6 percent). Compared internationally, the percentage of U.S. 16- to 19-year-olds with at least one parent attaining a high school degree scoring at the lowest proficiency level (24 percent) was larger than the OECD average for this same group (13 percent).

---

Figure 22.4. Percentage of 16- to 19-year-olds assessed in the problem solving in technology-rich environments domain performing at the lowest proficiency level (below level 1), by country: 2012, 2014, and 2015

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent.
¹ Data are from 2015.
² A supplemental round of data collection was completed in 2014 in order to expand the sample of U.S. adults, allowing for more in-depth data analysis.
³ Refers to the mean of the data values for all reporting Organization for Economic Cooperation and Development (OECD) countries and subnational education systems, to which each country or subnational education system reporting data contributes equally.
NOTE: Program for International Assessment of Adult Competencies (PIAAC) results for the United States are based on combined data from 2012 and 2014. Results for the OECD were mostly collected in 2012, but a second round of data collection was completed in 2015 for countries that did not participate in 2012. The proficiency levels correspond to the following score ranges on a scale of 0 to 500: below level 1 (0–240.9), level 1 (241.0–290.9), level 2 (291.0–340.9), and level 3 (341.0–500.0). Tasks at a higher level are more demanding in terms of requirements such as using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher level forms of reasoning. For detailed descriptions of each proficiency level, as well as specific examples of tasks at each level, see appendix B of the report Skills of U.S. Unemployed, Young, and Older Adults in Sharper Focus (NCES 2016-039rev), available at http://nces.ed.gov/pubs2016/2016039rev.pdf. Seven percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 to 34 percent across OECD countries.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for the International Assessment of Adult Competencies (PIAAC), U.S. PIAAC 2012/2014; Organization for Economic Cooperation and Development, PIAAC 2012 and 2015. See Digest of Education Statistics 2016, table 604.50.

---

Among the 25 OECD countries that reported problem solving in technology-rich environments scale scores, the percentages of 16- to 19-year-olds performing at the lowest proficiency level ranged from 2 percent in the Republic of Korea to 37 percent in Turkey. The percentage of 16- to 19-year-olds performing at the lowest level in the United States (18 percent) was larger than the percentages in 18 OECD countries, smaller than the percentages in 3 OECD countries (Turkey, Greece, and Chile) and not measurably different from the percentages in 3 OECD countries (Ireland, Israel, and Poland).
Three non-OECD countries (Lithuania, the Russian Federation, and Singapore) reported scores on the problem solving in technology-rich environments scale. The United States had a larger percentage of 16- to 19-year-olds scoring at the lowest proficiency level than did Singapore (18 vs. 7 percent) and no measurable difference compared with Lithuania and the Russian Federation.

---

Figure 22.5. Percentage of 16- to 19-year-olds assessed in the problem solving in technology-rich environments domain performing at the highest proficiency level (level 3), by country: 2012, 2014, and 2015

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent.
‡ Reporting standards not met. Either there are too few cases for a reliable estimate or the coefficient of variation (CV) is 50 percent or greater.
¹ Data are from 2015.
² Refers to the mean of the data values for all reporting Organization for Economic Cooperation and Development (OECD) countries and subnational education systems, to which each country or subnational education system reporting data contributes equally.
³ A supplemental round of data collection was completed in 2014 in order to expand the sample of U.S. adults, allowing for more in-depth data analysis.
NOTE: Program for International Assessment of Adult Competencies (PIAAC) results for the United States are based on combined data from 2012 and 2014. Results for the OECD were mostly collected in 2012, but a second round of data collection was completed in 2015 for countries that did not participate in 2012.  The proficiency levels correspond to the following score ranges on a scale of 0 to 500: below level 1 (0–240.9), level 1 (241.0–290.9), level 2 (291.0–340.9), and level 3 (341.0–500.0). Tasks at a higher level are more demanding in terms of requirements such as using specific as well as generic technology applications, using multiple functions and navigation, performing a greater number of steps, generating subgoals, evaluating information, and applying higher level forms of reasoning. For detailed descriptions of each proficiency level, as well as specific examples of tasks at each level, see appendix B of the report Skills of U.S. Unemployed, Young, and Older Adults in Sharper Focus (NCES 2016-039rev), available at http://nces.ed.gov/pubs2016/2016039rev.pdf. Seven percent of U.S. 16- to 19-year-olds were not assessed in problem solving in technology-rich environments. The percentage of 16- to 19-year-olds who were not assessed in problem solving in technology-rich environments ranged from 2 to 34 percent across OECD countries.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for the International Assessment of Adult Competencies (PIAAC), U.S. PIAAC 2012/2014; Organization for Economic Cooperation and Development, PIAAC 2012 and 2015. See Digest of Education Statistics 2016, table 604.50.

---

The percentages of 16- to 19-year-olds performing at the highest proficiency level of the problem solving in technology-rich environments scale in OECD countries ranged from 2 percent in Chile to 13 percent in New Zealand.⁴ The percentage of 16- to 19-year-olds performing at this level in the United States (4 percent) was smaller than the percentages in 6 OECD countries, and not measurably different from the percentages in 15 countries. With respect to non-OECD countries, the United States had a smaller percentage of 16- to 19-year-olds scoring at the highest proficiency level than did Singapore (14 percent), and no measurable difference compared with the percentage for the Russian Federation.⁵

---

¹ France, Italy, and Spain are omitted because these OECD countries did not assess problem solving in technology-rich environments.
² For more information, visit https://nces.ed.gov/surveys/piaac/pstreproficiencylevel.asp.
³ OECD average refers to the mean of the data values for all reporting OECD countries and subnational regions, to which each country or subnational region reporting data contributes equally.
⁴ Data for Greece, Northern Ireland (UK), and Turkey were not available because either there were too few cases for a reliable estimate or the coefficient of variation (CV) was 50 percent or greater.
⁵ Data for Lithuania were not available because either there were too few cases for a reliable estimate or the coefficient of variation (CV) was 50 percent or greater.

---