Skip Navigation
Illustration/Logo View Quarterly by  This Issue  |  Volume and Issue  |  Topics
Education Statistics Quarterly
Vol 4, Issue 1, Topic: International Statistics
Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy
By: Mariann Lemke, Christopher Calsyn, Laura Lippman, Leslie Jocelyn, David Kastberg, Yan Yun Liu, Stephen Roey, Trevor Williams, Thea Kruger, and Ghedam Bairu
 
This article was originally published as Highlights From the 2000 Program for International Student Assessment (PISA), a brochure summarizing key findings from the Outcomes of Learning report cited above. The sample survey data are from PISA.
 
 

Introduction

The Program for International Student Assessment (PISA) is a new system of international assessments that focus on 15-year-olds’ capabilities in reading literacy, mathematics literacy, and science literacy. PISA also measures general or cross-curricular competencies such as learning strategies.

PISA will be implemented on a 3-year cycle that began in 2000. Each PISA assessment cycle focuses on one particular subject, although all three are assessed in each cycle. In this first cycle, PISA 2000, reading literacy is the major focus, occupying roughly two-thirds of assessment time. In 2003, PISA will focus on mathematics literacy, and in 2006, on science literacy (figure 1).

PISA will report on performance in reading literacy, mathematics literacy, and science literacy every 3 years, and provide a more detailed look at each domain in the years when it is the major focus. For instance, average scores for specific reading processes, such as retrieving information, interpreting texts, and reflecting on texts, as well as a combined reading literacy average score are available for PISA 2000. Only single measures of mathematics and science literacy are available in PISA 2000, with more specific information to be provided for these domains in subsequent cycles. These cycles will allow countries to compare changes in trends for each of the three content areas over time. Future cycles will also include further development of the assessment of cross-curricular competencies, such as problem solving in 2003 and use of information and communications technology in 2006.

PISA is sponsored by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of 30 industrialized nations. In 2000, 32 countries participated in PISA, including 28 OECD countries and 4 non-OECD countries (figure 2).

PISA’s purpose is to represent the overall yield of learning for 15-year-olds. PISA assumes that by the age of 15, young people have had a series of learning experiences, both in and out of school, that allow them to perform at particular levels in reading, mathematics, and science literacy. Other national and international studies have a strong link to curriculum frameworks and seek to measure students’ mastery of specific knowledge, skills, and concepts. PISA is designed to measure “literacy” more broadly. The unique contribution of PISA lies in its focus on assessing students’ knowledge and skills in reading, mathematics, and science in the context of everyday situations. As 15-year-olds begin to make the transition to adult life, they not only need to know how to read, or understand particular mathematical formulas or scientific concepts, but they also need to be able to apply knowledge and skills in all of the different situations they will encounter in their lives. By focusing on the age of 15, PISA allows countries to compare outcomes of learning that reflect both societal and educational system influences, as well as students’ preparedness for adult life as they near the end of compulsory schooling.

The United States has been actively involved in the development of PISA since its inception, believing that PISA’s differences from other studies allow it to complement the picture of U.S. performance obtained from other studies and provide a new perspective on U.S. education in an international context. This report, produced by the U.S. Department of Education’s National Center for Education Statistics (NCES), focuses on U.S. results from PISA 2000. Following are highlights of the findings presented in the complete report.

Figure 1.—Program for International Student Assessment (PISA) assessment cycle
Figure 1.- Program for International Student Assessment (PISA) assessment cycle

NOTE: The subject in all capital letters in each assessment cycle is the major domain for that cycle.

SOURCE: Previously published as figure 1 on p. 2 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

back to top


Reading Literacy

PISA measures how well 15-year-olds are able to apply different reading processes to a wide range of reading materials, such as the kinds of forms they receive from their governments, the kinds of articles they read in their local newspapers, the kinds of manuals they read for work or school, or the kinds of books or magazines they read for entertainment.

PISA scores are reported on a scale of 0 to 1,000, with a mean of 500 and a standard deviation of 100. Most scores fall between 200 and 800. The three specific reading processes on which PISA 2000 reports are

  • retrieving information—the ability to locate one or more pieces of information in a text;
  • interpreting texts—the ability to construct meaning and draw inferences from one or more parts of a text; and
  • reflecting on texts—the ability to relate a text to one’s own experience, knowledge, and ideas.
Average subscale scores are reported for each of these three reading processes. Together, these three subscale scores make up the combined reading literacy score.

Figure 2.—Participating countries in the Program for International Student Assessment (PISA) 2000
Figure 2.- Participating countries in the Program for International Student Assessment (PISA) 2000

NOTE: Although the Netherlands participated in the Program for International Student Assessment (PISA) in 2000, technical problems with its sample prevent its results from being discussed here. For information on the results for the Netherlands, see Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000 (OECD 2001).

SOURCE: Previously published as figure 2 on p. 3 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

National averages
  • On the combined reading literacy scale for PISA 2000, U.S. 15-year-olds perform about as well on average as 15-year-olds in most of the 27 participating OECD countries. Students in Canada, Finland, and New Zealand outperform U.S. students. U.S. students perform at the same level as students in 19 other participating OECD countries and Liechtenstein. U.S. students perform better on average than students from the OECD nations of Greece, Luxembourg, Mexico, and Portugal (figure 3).
  • For each of the three specific reading process subscales, retrieving information, interpreting texts,and reflecting on texts, U.S. scores are not different from the OECD averages. Canada and Finland outscore the United States on each of the three reading process subscales, and the United States outscores at least seven other nations on each measure.
  • There are clear consistencies across the three reading process subscales of retrieving information, interpreting texts, and reflecting on texts, which carry through to the combined reading literacy score.
  • Fifteen countries, or about half of the countries participating in PISA 2000, show less variation in student performance than the United States. The remaining countries show similar variation in student performance to the United States, and U.S. variation is similar to the OECD average.
  • The top 10 percent of OECD students score 623 or higher on the combined reading literacy scale. In the United States, 13 percent of students achieve this score or better, a percentage not different from the OECD top 10 percent benchmark. Three countries (Canada, Finland, and New Zealand) have a higher percentage of students scoring in the OECD top 10 percent, while 14 countries have a lower percentage.
Figure 3.—Combined reading literacy average scores and average subscale scores of 15-year-olds, by country: 2000
Figure 3.- Combined reading literacy average scores and average subscale scores of 15-year-olds, by country: 2000
NOTE: Although the Netherlands participated in the Program for International Student Assessment (PISA) in 2000, technical problems with its sample prevent its results from being discussed here. For information on the results for the Netherlands, see Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000 (OECD 2001). The OECD average is the average of the national averages of 27 OECD countries. Because PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries and not included in the OECD average.

SOURCE: Previously published as figure 3 on p. 11 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

Reading literacy levels

PISA uses five levels to describe student performance in reading literacy. In order to reach a particular level, a student must be able to answer correctly a majority of items at that level. The percentage of 15-year-olds at each level of reading literacy for participating countries is shown in figure 4.

  • Percentages of U.S. students across the literacy levels are similar to the OECD average percentages, except at level 5. In the United States, 12 percent of 15-year-olds read at level 5, the highest proficiency level, a percentage higher than the OECD average. Level 1 encompasses 12 percent of U.S. students, and 6 percent of U.S. 15-year-olds are below level 1.
  • Percentages of U.S. 15-year-olds across the levels for the three reading process subscales are consistent with the percentages for the combined reading literacy scale. That is, about 12 percent of U.S. 15-year-olds are at level 5 for retrieving information, interpreting texts, and reflecting on texts, and for the combined reading literacy scale; about 21 percent are at level 4 for these three subscales and the combined reading literacy scale; and so on.
  • Looking at the cumulative percentages of students from level to level on the combined reading literacy scale, about one-third of U.S. students perform at the two highest levels, level 4 and level 5. About 60 percent of students in the United States perform at level 3 or above, and over 80 percent at level 2 or above.
back to top


Mathematics and Science Literacy

PISA’s mathematics and science literacy assessments focus on 15-year-olds’ abilities to apply mathematical and scientific principles and thinking in a wide variety of situations. Figure 5 displays national averages in mathematics and science literacy.

  • In both mathematics and science literacy, the U.S. average does not differ from the OECD average. Eight countries outperform the United States in mathematics literacy, and seven have higher average scores for science literacy. The United States has higher average scores than seven countries for mathematics literacy and seven for science literacy.
  • The top 10 percent of students in OECD countries score 625 or higher in mathematics literacy. In the United States, 9 percent of students achieve this score or better, a percentage not different from the OECD top 10 percent benchmark. Eight countries have a greater proportion of students scoring in the OECD top 10 percent, while six countries have a smaller proportion.
  • For science literacy, the top 10 percent of OECD students score 627 or higher. In the United States, 10 percent of students achieve this score or better. Four countries have a higher percentage of students scoring in the OECD top 10 percent, while seven countries have a lower percentage.
back to top


Demographic Profiles of Reading, Mathematics, and Science Literacy

In the United States and many other countries, policy-makers are not only interested in overall achievement but also in achievement by specific population groups.

  • On the combined reading literacy scale, female 15-year-olds outperform male 15-year-olds in every country. On the PISA 2000 mathematics literacy assessment, performance of males and females in the United States is similar, as it is in 16 other countries; 14 countries show higher performance for males than females for mathematics literacy. For most countries (26 out of 31 countries), including the United States, males and females perform similarly on the science literacy assessment (figure 6).
  • In the United States, parents’ education is strongly linked to differences in student performance in reading, mathematics, and science literacy.
  • In the United States, the relationship of socioeconomic status to literacy levels is about the same for each subject. Increases in socioeconomic status are associated with increases in scores for reading literacy, mathematics literacy, and science literacy. Most participating countries do not differ significantly from the United States in terms of the strength of the relationship between socioeconomic status and literacy in any subject.
  • In the United States, parents’ national origin is linked to performance in reading literacy and mathematics literacy only for those students with two foreign-born parents compared with students with two native-born parents. There is no difference in science literacy achievement between students with native- and foreign-born parents.
  • In the United States, 89 percent of students report that they speak the language of the assessment (English) at home most of the time. In the United States and most other countries, the reading literacy achievement of students who speak the test language at home is higher than that of students not speaking this language at home. The United States and most other countries also show advantages for test-language speakers in mathematics and science literacy.
  • The pattern of between-group differences for racial and ethnic groups in the United States is identical across the three literacy areas. In reading, mathematics, and science, the average literacy scores for Whites and “other”* students are higher than for Hispanic and Black students.
Figure 4.—Percentage distribution of 15-year-olds by combined reading literacy scores, by level and by country: 2000
Figure 4.- Percentage distribution of 15-year-olds by combined reading literacy scores, by level and by country: 2000
NOTE: The Program for International Student Assessment (PISA) uses five levels of performance to describe student performance. In order to reach a particular level, a student must be able to correctly answer a majority of items at that level. Students were classified into reading levels according to their scores. Although the Netherlands participated in PISA in 2000, technical problems with its sample prevent its results from being discussed here. For information on the results for the Netherlands, see Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000 (OECD 2001). The OECD average is the average of the national averages of 27 OECD countries. Because PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries and not included in the OECD average.

SOURCE: Previously published as figure 8 on p. 19 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

Figure 5.—Mathematics and science literacy average scores of 15-year-olds, by country: 2000
Figure 5.- Mathematics and science literacy average scores of 15-year-olds, by country: 2000

NOTE: Although the Netherlands participated in the Program for International Student Assessment (PISA) in 2000, technical problems with its sample prevent its results from being discussed here. For information on the results for the Netherlands, see Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000 (OECD 2001). The OECD average is the average of the national averages of 27 OECD countries. Because PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries and not included in the OECD average.

SOURCE: Previously published as figure 10 on p. 26 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

Figure 6.—Differences in average scores in reading, mathematics, and science literacy of 15-year-olds by gender, by country: 2000
Figure 6.- Differences in average scores in reading, mathematics, and science literacy of 15-year-olds by gender, by country: 2000

NOTE: Each bar above represents the average score difference between males and females on combined reading, mathematics, or science literacy. Some of these differences are statistically significant and indicated by darker bars. For instance, the United States has a 29-point score difference favoring females in combined reading literacy, which is statistically significant. The score differences between U.S. males and females in mathematics and science literacy are 7 points and 5 points, respectively, but neither is a statistically significant difference. Average score difference is calculated by subtracting scores of males from scores of females. Detail may not add to totals because of rounding. Although the Netherlands participated in the Program for International Student Assessment (PISA) in 2000, technical problems with its sample prevent its results from being discussed here. For information on the results for the Netherlands, see Knowledge and Skills for Life: First Results from the OECD Programme for International Student Assessment (PISA) 2000 (OECD 2001). The OECD average is the average of the national averages of 27 OECD countries. Because PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries and not included in the OECD average.

SOURCE: Previously published as figure 15 on p. 39 of the complete report that this article summarizes (Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy [NCES 2002–115]).

back to top


A First Step in Cross-Curricular Competencies

One of PISA’s main objectives is to measure student performance on general or nonacademic learning outcomes in addition to outcomes for reading, mathematics, and science literacy. As a first step toward the measurement of cross-curricular competencies, in PISA 2000, student questionnaire items sought information in two major areas, student attitudes toward reading and learning strategies.

  • Thirty percent of U.S. 15-year-olds agree or strongly agree that reading is a favorite hobby, a lower percentage than the OECD average. Percentages of students agreeing that reading is a favorite hobby range from 62 percent in Mexico to 24 percent in Norway.

  • In every country, females agree more frequently than males that reading is a favorite hobby. Thirty-seven percent of females in the United States agree that reading is a favorite hobby, compared to 22 percent of males.

  • About half of U.S. 15-year-olds report trying to memorize as much as possible often or always when studying. The U.S. percentage in this case is higher than the OECD average, suggesting that a greater proportion of U.S. students often use memorization as a learning strategy than the average proportion of OECD country students.

  • The percentages of students who respond that they often or always try to relate new material to things they have already learned range from 15 percent in Italy to 90 percent in Hungary. Fifty-nine percent of U.S. students report using this strategy frequently, a higher percentage than the OECD average.
back to top


Footnote

*

The “other” group comprises students identifying themselves as American Indian/Alaska Native, Native Hawaiian/Pacific Islander, or multiracial.

back to top


Reference

Organization for Economic Cooperation and Development. (2001). Knowledge and Skills for Life: First Results From the OECD Programme for International Student Assessment (PISA) 2000. Paris: Author.

back to top


Data source: The Program for International Student Assessment (PISA), 2000.

For technical information, see the complete report:

Lemke, M., Calsyn, C., Lippman, L., Jocelyn, L., Kastberg, D., Liu, Y.Y., Roey, S., Williams, T., Kruger, T., and Bairu, G. (2001). Outcomes of Learning: Results From the 2000 Program for International Student Assessment of 15-Year-Olds in Reading, Mathematics, and Science Literacy (NCES 2002–115).

Author affiliations: M. Lemke, L. Lippman, and G. Bairu, NCES; C. Calsyn and T. Kruger, ESSI; L. Jocelyn, D. Kastberg, Y.Y. Liu, S. Roey, and T. Williams, Westat, Inc.

For questions about content, contact pisa@ed.gov.

To obtain the complete report (NCES 2002–115), call the toll-free ED Pubs number (877–433–7827) or visit the NCES Web Site (http://nces.ed.gov).

To obtain the Highlights brochure used for this article (NCES 2002–116), call the toll-free ED Pubs number (877–433–7827) or visit the NCES Web Site (http://nces.ed.gov).

back to top