• Key Findings
• Opportunities for Related Research
• Summary
• References

The problems inherent in improving mathematics and science education are complex, and, while the most prominent features of international comparative studies are often the achievement results and international rankings, it is important to note that data on achievement are not sufficient for capturing this complexity. The additional contextual information available in the TIMSS and PISA reports-information about the performance of particular population groups, as defined by socioeconomic status and other variables-is helpful in making interpretations and conjectures about these problems. The studies alone can point us to some descriptive conclusions about the present state of affairs; an even greater contribution is the wealth of ideas they can provide that warrant more specific exploration and study. The involvement of mathematics and science researchers and K-12 practitioners-in collaboration with mathematicians, scientists, and psychometricians-in interpreting these reports, undertaking secondary analyses, and conceptualizing related follow-up studies is a promising pathway for making optimal use of the resources provided in the TIMSS and PISA suite of studies. Using mathematics as the context, the following discussion provides some highlights of the findings and some ideas about follow-up work for researchers and practitioners (see Ferrini-Mundy and Schmidt [2005] for additional discussion). Some findings from the problem-solving component of PISA are presented as well.

U.S. mathematics performance in TIMSS 2003

To interpret the TIMSS 2003 U.S. mathematics results, one needs to consider U.S. performance relative to that of other countries in 2003-as well as since 1995-and changes in U.S. performance since 1995.¹ In 2003, the scores of U.S. fourth- and eighth-graders on the mathematics component of TIMSS exceeded the international averages, with students in both grades outperforming a little more than half of their peers in the other participating countries (13 of 24 countries at grade 4; 25 of 44 countries at grade 8). In the content areas, U.S. fourth- and eighth-graders performed above the international averages in four of the five mathematics content areas measured in 2003.² U.S. eighth-graders also demonstrated significant improvement between 1999 and 2003 in two of the content areas-algebra and data. Looking at population groups, U.S. eighth-grade boys, girls, Blacks, and Hispanics improved their mathematics performance between 1995 and 2003-and the improved performance of Black eighth-graders resulted in a narrowing of the gap in achievement between White and Black students. The data suggest that the improved performance of U.S. eighth-graders in mathematics, combined with a decline in performance among some of the countries that were the top performers in the earlier assessments, resulted in a higher relative standing of U.S. eighth-graders in 2003 compared to 1995.

Whereas U.S. eighth-graders showed improved performance in mathematics between 1995 and 2003, U.S. fourth-graders showed no measurable improvement. Between 1995 and 2003, fourth-graders in 6 of the 15 countries participating in TIMSS 2003 and TIMSS 1995 demonstrated improved performance, but the United States was among a set of seven countries whose students' scores did not change measurably over the period (Gonzales et al. 2004, p. 6). Moreover, the data suggest that the performance of U.S. fourth-graders in mathematics was lower in 2003 than in 1995 relative to the 14 other countries that participated in both studies. Students in two countries (England and Latvia-LSS³) who performed below U.S. fourth-graders in mathematics in 1995 improved their performance to such a degree that they outperformed U.S. students in 2003. The one positive finding at grade 4 was that U.S. Black students improved their mathematics performance between 1995 and 2003, resulting in a narrowing of the achievement gap between White and Black students (Gonzales et al. 2004).

U.S. mathematics literacy and problem-solving performance in PISA 2003

In contrast to TIMSS, which is part of a long-standing series of international comparative assessments that have sought to measure achievement as it relates to the implemented and intended curriculum, PISA is not so explicitly focused on curricular issues. Rather, it "provides a unique and complementary perspective" by focusing "on the application of knowledge in reading, mathematics, and science to problems with a real-life context" (OECD 1999, cited in Lemke et al. 2004, p. 2). By choosing to address mathematics literacy, and by providing the associated definition⁴ together with an accompanying conceptual framework, PISA has made a major contribution to the international comparative literature in mathematics.

In PISA 2003, the U.S. averages in mathematics literacy and problem solving were lower than the averages for most OECD countries. However, in mathematics literacy, the United States showed no measurable changes in the trend areas of space and shape and change and relationships from the 2000 administration of PISA. Comparisons in performance by gender, race/ethnicity, and socioeconomic status are also available. In mathematics literacy, U.S. males outperformed females at the high proficiency levels, although females were not overrepresented at the lowest proficiency levels. In problem solving, there were no gender differences in performance for U.S. students. Across proficiency levels in mathematics literacy and problem solving among U.S. students, the performance of Blacks and Hispanics was generally lower than that of students who were White, Asian, or of more than one race.

Some commentators have viewed the mathematics results of TIMSS 2003 and PISA 2003 as conflicting, with a sense that the TIMSS results are more positive than the results for PISA. The TIMSS results may suggest that U.S. students' performance is at least holding steady, and possibly improving, in relatively traditional school mathematics content. PISA is explicit about testing knowledge that may not be part of the intended curriculum and that reflects students' ability to apply mathematics in context. U.S. standards-based reforms in mathematics education in the past two decades have emphasized the importance of being able to use mathematics in real-world situations; U.S. students' performance in PISA thus provides a much-needed baseline for future examination of the ability of U.S. students to apply mathematics in context.

The findings presented here, as well as others in the TIMSS and PISA reports, suggest that additional analyses of curriculum and the opportunity to learn might help to explain the differences in performance in certain subareas of TIMSS and PISA. In both assessments, the patterns of differential performance by specific subgroups and in specific subareas might lead researchers to revisit conjectures they have had about the nature of the curricular (and extracurricular) opportunities that are afforded to students to use mathematics. The conceptual framework of TIMSS 2003 (Mullis et al. 2003) (which closely mirrors the National Assessment of Educational Progress framework) and that of PISA 2003 (OECD 2003) can be helpful tools for researchers conducting alignment analyses of instructional materials and state frameworks similar to those performed by Schmidt and colleagues (e.g., Schmidt et al. 1997; Schmidt et al. 2001) using the TIMSS 1995 curriculum framework (Robitaille et al. 1993). A snapshot of the nature of curriculum and the nature of instruction available to students would provide additional useful context for interpretation. The uneven progress overall in the 8 years between TIMSS 1995 and TIMSS 2003 may indicate that efforts to improve mathematics performance in the United States lack collective focus, coherence, intensity, or scale-and that continued efforts to make improvements, together with research designed to understand the interventions and track their impact, are needed. The findings in these reports suggest the importance of secondary analyses and additional studies that probe in detail to better understand the contexts, conditions, and interventions that may have been at play in settings where improvement in achievement is occurring and the achievement gap is narrowing.

In summary, international comparative research such as that of TIMSS and PISA is essential for developing a better understanding of the state of U.S. mathematics and science education. The periodic opportunity to benchmark U.S. performance against that of countries around the world provides the impetus needed for an ongoing examination of all aspects of the U.S. education system, including curriculum, instruction, and teacher education and preparation. In addition, the ongoing development of conceptual frameworks such as those produced for TIMSS 2003 and PISA 2003-which push collective thinking about what is appropriate in the K-12 curriculum for mathematics and science-is a welcome contribution to the literature in mathematics and science education that comes from a source outside of the U.S. education community-the international comparative studies community.

Ferrini-Mundy, J., and Schmidt, W.H. (2005). International Comparative Studies in Mathematics Education: Opportunities for Collaboration and Challenges for Researchers. Journal for Research in Mathematics Education, 36(3): 164-175.

Gonzales, P., Guzmán, J.C., Partelow, L., Pahlke, E., Jocelyn, L., Kastberg, D., and Williams, T. (2004). Highlights From the Trends in International Mathematics and Science Study (TIMSS) 2003 (NCES 2005-005). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Lemke, M., Sen, A., Pahlke, E., Partelow, L., Miller, D., Williams, T., Kastberg, D., and Jocelyn, L. (2004). International Outcomes of Learning in Mathematics Literacy and Problem Solving: PISA 2003 Results From the U.S. Perspective (NCES 2005-003). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., and Chrostowski, S.J. (2004). TIMSS 2003 International Mathematics Report: Findings From IEA's Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.

Mullis, I.V.S., Martin, M.O., Smith, T.A., Garden, R.A., Gregory, K.D., Gonzalez, E.J., Chrostowski, S.J., and O'Connor, K.M. (2003). TIMSS Assessment Frameworks and Specifications 2003. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.

Organization for Economic Cooperation and Development. (2004a). Learning for Tomorrow's World: First Results From PISA 2003. Paris: Author.

Organization for Economic Cooperation and Development. (2004b). Problem Solving for Tomorrow's World: First Measures of Cross-Curricular Competencies From PISA 2003. Paris: Author.

Organization for Economic Cooperation and Development. (2003). The PISA 2003 Assessment Framework-Mathematics, Reading, Science and Problem Solving Knowledge and Skills. Paris: Author.

Organization for Economic Cooperation and Development. (1999). Measuring Student Knowledge and Skills: A New Framework for Assessment. Paris: Author.

Robitaille, D.F., Schmidt, W.H., Raizen, S., McKnight, C., Britton, E., and Nicol, C. (1993). Curriculum Frameworks for Mathematics and Science (TIMSS Monograph No. 1). Vancouver, Canada: Pacific Educational Press.

Schmidt, W.H., McKnight, C.C., Houang, R.T., Wang, H.C., Wiley, D.E., Cogan, L.S., and Wolfe, R.G. (2001). Why Schools Matter: A Cross-National Comparison of Curriculum and Learning. San Francisco: Jossey-Bass.

Schmidt, W.H., McKnight, C.C., Valverde, G.A., Houang, R.T., and Wiley, D.E. (1997). Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School Mathematics. Dordrecht: Kluwer Academic Publishers.

¹TIMSS 2003 is the third comparison of mathematics and science achievement carried out by the International Association for the Evaluation of Educational Achievement. Previously conducted in 1995 and 1999, TIMSS can be used to track changes in achievement over time.

² The five mathematics content areas measured in TIMSS 2003 were number, algebra (at the fourth grade, patterns and relationships), measurement, geometry, and data.

³Designated LSS because only Latvian-speaking schools were included in 1995. For this analysis, only Latvian-speaking schools are included in the 2003 average.

⁴Mathematics literacy is defined as "an individual's capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgments and to use and engage with mathematics in ways that meet the needs of that individual's life as a constructive, concerned, and reflective citizen" (OECD 2003, p. 24).