Skip Navigation
Data
Point
U.S. Department of Education NCES 2017-249 November 2017
Collaborative Problem Solving Skills of 15-Year-Olds: Results From PISA 2015

The Program for International Student Assessment (PISA) is a global education study of 15-year-old students' reading, mathematics, and science literacy and, in 2015, two optional components: financial literacy and collaborative problem solving.1 Fifty-one education systems administered the collaborative problem solving assessment, including 32 of 35 OECD-member education systems.2 Students were assessed on their ability to solve a problem by sharing the understanding and effort required to come to a solution, and pooling their knowledge, skills and effort to reach that solution (OECD 2017).3 Assessment results are reported on a scale of 01,000, and also in terms of the percentage of students performing at four proficiency levels.4,5

How did U.S. 15-year-olds perform, on average, on the PISA collaborative problem solving assessment in 2015?

Figure 1. Percentage distribution of 15-year-old students on the PISA collaborative problem solving proficiency levels and average scores, by education system: 2015

Figure 1. Percentage distribution of 15-year-old students on the PISA collaborative problem solving proficiency levels and average scores, by education system: 2015

Average score is higher than U.S average score at the .05 level of statisical significance. Average score is higher than U.S average score at the .05 level of statisical significance.
Average score is lower than U.S average score at the .05 level of statisical significance. Average score is lower than U.S average score at the .05 level of statisical significance.
# Rounds to zero.
! Interpret data with caution. Estimate is unstable due to high coefficient of variation (>30 percent and ≤50 percent).
‡ Reporting standards not met due to coefficient of variation over 50 percent.NOTE: Education systems are ordered by 2015 average score. Scores are reported on a scale from 0 to 1,000. To reach a particular proficiency level, a student must correctly answer a majority of items at that level. Students were classified into proficiency levels according to their scores. Cut scores are as follows: below level 1 (a score less than 340); level 1 (a score greater than or equal to 340 and less than 440); level 2 (a score greater than or equal to 440 and less than 540); level 3 (a score greater than or equal to 540 and less than 640); and level 4 (a score greater than or equal to 640). The OECD average is the average of the estimates of the 32 OECD countries that participated in the collaborative problem solving assessment, with each country weighted equally. The following OECD countries did not participate in the assessment: Ireland, Poland, and Switzerland. Italics indicate non-OECD countries and education systems. B-S-J-G (China) refers to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for public school students only. Standard error is noted by s.e. Detail may not sum to totals because of rounding.SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015.

  • The U.S. average score of 520 was higher than the OECD average of 500. Average scores ranged from 382 in Tunisia to 561 in Singapore (figure 1).
  • Students in 10 education systems had higher average scores than U.S. students, including 7 OECD education systems—Japan, Korea, Canada, Estonia, Finland, New Zealand, and Australia—and 3 other systems—Singapore, Hong Kong (China), and Macau (China).
  • Students in five education systems had average scores that were not measurably different than the U.S. average score: Chinese Taipei, Germany, Denmark, the United Kingdom, and the Netherlands. Of these, four are OECD systems. Students in 35 education systems had average scores lower than U.S. students, including 20 OECD systems.

What is the percentage distribution of U.S. students across the proficiency levels on the PISA collaborative problem solving assessment?

  • Students scoring at the highest proficiency level—level 4—completed complicated problem-solving tasks with high levels of collaboration complexity. Fourteen percent of U.S. 15-year-olds performed at proficiency level 4, which was higher than the OECD average of 8 percent (figure 1; see note below figure for details on the score ranges associated with each proficiency level).
  • Compared to the level 4 percentages of all education systems, the U.S. percentage was higher than in 40 education systems, not measurably different than in 9, and lower than in only one—Singapore. Among OECD systems, the U.S. percentage was higher than in 23 and not measurably different than in 8. No OECD system had a higher percentage of students performing at level 4 than the United States.
  • Some students who scored below proficiency level 2—considered below a baseline level of proficiency in collaborative problem solving—completed low-level problems with limited collaboration complexity, while others could not complete even low-level tasks.6 Nearly one-quarter of U.S. students performed below level 2, which was lower than the OECD average of 28 percent. Among all education systems, the percentage scoring below level 2 ranged from 10 percent in Japan to 84 percent in Tunisia.
  • The percentage of U.S. students performing below level 2 was lower than in 28 and higher than in 13 education systems. Among OECD systems, the percentage of U.S. students was lower than in 13 systems, not measurably different than in 9, and higher than in 9.

Endnotes

1 PISA is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries, and is conducted in the United States by NCES. More information about the PISA collaborative problem solving assessment is available at http://www.oecd.org/pisa.
2 PISA participants include countries and subnational regions such as states or provinces. For convenience, reference is made to "education systems" for all participants. Malaysia administered the collaborative problem solving assessment in 2015 but due to sampling issues, its data were deemed not comparable and are not shown here.
3 Refer to the PISA 2015 framework for additional details on the definition of collaborative problem solving, available at http://www.oecd-ilibrary.org/education/pisa-2015-assessment-and-analytical-framework_9789264281820-en;jsessionid=3m26bnac3umki.x-oecd-live-03.
4 Descriptions of the skills of students at each proficiency level can be found in exhibit CPS1, available at https://nces.ed.gov/surveys/pisa/pisa2015/index.asp.
5 Standard errors for the estimates are available at https://nces.ed.gov/surveys/pisa/pisa2015/index.asp.
6 Below level 2 is a combination of students performing at level 1 and below level 1 (shown as level 1 and below in figure 1).

Reference

Organization for Economic Cooperation and Development (OECD). (2017). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. Paris: Author.

Data in this report are from the Program for International Student Assessment (PISA), an international assessment of 15-year-old students, a sample-based survey. To learn more about PISA, visit http://nces.ed.gov/surveys/pisa.

This NCES Data Point presents information on education topics of current interest. It was authored by David Kastberg of Westat and Anindita Sen of the American Institutes for Research. Estimates based on samples are subject to sampling variability, and apparent differences may not be statistically significant. All noted differences are statistically significant at the .05 level. In the design, conduct, and data processing of National Center for Education Statistics (NCES) surveys, efforts are made to minimize the effects of nonsampling errors, such as item nonresponse, measurement error, data processing error, or other systematic error.