Skip Navigation
Illustration/Logo View Quarterly by  This Issue  |  Volume and Issue  |  Topics
Education Statistics Quarterly
Vol 6, Issue 4, Topic: Featured Topic: International Assessments
Highlights From the Trends in International Mathematics and Science Study (TIMSS) 2003
By: Patrick Gonzales, Juan Carlos Guzmán, Lisette Partelow, Erin Pahlke, Leslie Jocelyn, David Kastberg, and Trevor Williams
 
This article was originally published as the Introduction and Summary of the Statistical Analysis Report of the same name. The sample survey data are from the Trends in International Mathematics and Science Study (TIMSS).  
 
 

Introduction

The 2003 Trends in International Mathematics and Science Study (TIMSS) is the third comparison of mathematics and science achievement carried out since 1995 by the International Association for the Evaluation of Educational Achievement (IEA), an international organization of national research institutions and governmental research agencies. TIMSS can be used to track changes in achievement over time. Moreover, TIMSS is closely linked to the curricula of the participating countries, providing an indication of the degree to which students have learned concepts in mathematics and science they have encountered in school. In 2003, some 46 countries participated in TIMSS, at either the fourth- or eighth-grade level, or both.

This summary highlights initial findings on the performance of U.S. fourth- and eighth grade students relative to their peers in other countries on the TIMSS assessment. The summary is based on the findings presented in two reports published by the IEA:

  • TIMSS 2003 International Mathematics Report: Findings From IEA's Trends in International Mathematics and Science Study at the Eighth and Fourth Grades (Mullis et al. 2004); and
  • TIMSS 2003 International Science Report: Findings From IEA's Trends in International Mathematics and Science Study at the Eighth and Fourth Grades (Martin et al. 2004).
These two IEA reports were simultaneously published with this summary report and are available online at http://www.timss.org.

This summary report describes the mathematics and science performance of fourth- and eighth-graders in participating countries over time. For a number of the participating countries, changes in mathematics and science achievement can be documented over 8 years, from 1995 to 2003. For others, changes can be documented over a shorter period of time, 4 years (from 1999 to 2003). Table A shows the countries that participated in TIMSS 2003, and their participation in earlier TIMSS data collections.* The fourth-grade assessment was offered in 1995 and 2003, while the eighth-grade assessment was offered in 1995, 1999, and 2003.

Average student performance in the United States is compared to that of students in other countries that participated in each assessment:

  • At fourth grade, comparisons are made among students in the 25 countries that participated in TIMSS 2003, and in the 15 countries that participated in TIMSS 2003 and TIMSS 1995.
  • At eighth grade, comparisons are made among students in the 45 countries that participated in TIMSS 2003, and in the 34 countries that participated in TIMSS 2003 and at least one earlier data collection, either TIMSS 1995 or 1999, or both.
  • All estimates for the United States are based on the performance of students from both public and private schools, unless otherwise indicated.
All countries that participated in TIMSS 2003 were required to draw random, nationally representative samples of students and schools. The U.S. fourth-grade sample achieved an initial school response rate of 70 percent (weighted), with a school response rate of 82 percent after replacement schools were added. From the schools that agreed to participate, students were sampled in intact classes. A total of 10,795 fourth-grade students were sampled for the assessment and 9,829 participated, for a 95 percent student response rate. The resulting fourth-grade overall response rate, with replacements included, was 78 percent. The U.S. eighth-grade sample achieved an initial school response rate of 71 percent, with a school response rate of 78 percent after replacement schools were added. A total of 9,891 students were sampled for the eighth-grade assessment and 8,912 completed the assessment, for a 94 percent student response rate. The resulting eighth-grade overall response rate, with replacements included, was 73 percent.

In addition to the assessments, students, their teachers, and principals were asked to complete questionnaires related to their school and learning experiences. At fourth grade, the assessment took approximately 72 minutes to complete. At eighth grade, the assessment took approximately 90 minutes. Detailed information on data collection, sampling, response rates, test development and design, weighting, and scaling is included in appendix A of the full report. Example items from the fourth- and eighth-grade assessments are included in appendix B of the full report.

Comparisons made in this report have been tested for statistical significance at the .05 level. Differences between averages or percentages that are statistically significant are discussed using comparative terms such as "higher" and "lower." Differences that are not statistically significant are either not discussed or referred to as "no measurable differences found" or "not statistically significant." In this latter case, failure to find a statistically significant difference should not be interpreted to mean that the estimates are the same or similar; rather, failure to find a difference may also be due to measurement or sampling error. Information on the technical aspects of the study can be found in appendix A of the full report, as well as in the TIMSS 2003 Technical Report (Martin, Mullis, and Chrostowski 2004).

Detailed tables with estimates and standard errors for all analyses included in this report are provided in appendix C of the full report. A list of TIMSS publications and resources published by the National Center for Education Statistics (NCES) and the IEA is provided in appendix E of the full report.


Table A. Participation in the TIMSS fourth-grade and eighth-grade assessments, by country: 1995, 1999, and 2003

Table A. Participation in the TIMSS fourth-grade and eighth-grade assessments, by country: 1995,  1999, and 2003
1Because of national-level changes in the starting age/date for school, 1999 data for Australia and Slovenia cannot be compared to 2003.

2Only the Flemish education system in Belgium participated in TIMSS in 2003.

3England collected data at grade 8 in 1995, 1999, and 2003, but due to problems with meeting the minimum sampling requirements for 2003, its eighth-grade data are not shown in this report.

4Hong Kong is a Special Administrative Region (SAR) of the People's Republic of China.

5Because of changes in the population tested, 1995 data for Israel and Italy, and 1999 data for Morocco are not shown.

6Only Latvian-speaking schools were included in 1995 and 1999. For trend analyses, only Latvian-speaking schools are included in the estimates.

7Because within-classroom sampling was not accounted for, 1995 data are not shown for South Africa.

NOTE: Countries that participated in 1995 and 1999 but did not participate in 2003 are not shown. Only countries that completed the necessary steps for their data to appear in the reports from the International Study Center are listed. In addition to the countries listed above, four separate jurisdictions participated in TIMSS 2003: the provinces of Ontario and Quebec in Canada; the Basque region of Spain; and the state of Indiana. Information on these four jurisdictions can be found in the international TIMSS 2003 reports. The Syrian Arab Republic participated in TIMSS 2003 at the eighth-grade level, but due to sampling difficulties, it is not shown in this report. Yemen participated in TIMSS 2003 at the fourth-grade level, but because it did not comply with the minimum sample requirements, it is not shown in this report. Countries could participate at either grade level. Countries were required to sample students in the upper of the two grades that contained the largest number of 9-year-olds and 13-year-olds, respectively. In the United States and most countries, this corresponds to grade 4 and grade 8. See table A1 in appendix A of the full report for details.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995, 1999, and 2003. (Originally published as table 1 on pp. 1-2 of the complete report from which this article is excerpted.)

Back to top


Summary

Looking across the results in mathematics and science, the following points can be made:

  • In 2003, fourth-graders in 3 countries-Chinese Taipei, Japan, and Singapore-outperformed U.S. fourth-graders in both mathematics and science, while students in 13 countries turned in lower average mathematics and science scores than U.S. students. U.S. fourth-grade students outperformed their peers in five OECD member countries (Australia, Italy, New Zealand, Norway, and Scotland), of which three are English-speaking countries (Australia, New Zealand, and Scotland).
  • No measurable changes were detected in the average mathematics and science scores of U.S. fourth-graders between 1995 and 2003. Moreover, the available data suggest that the performance of U.S. fourth-graders in both mathematics and science was lower in 2003 than in 1995 relative to the 14 other countries that also participated in both studies (tables B and C).
  • On the other hand, fourth-grade students in six countries showed improvement in both average mathematics and science scores between 1995 and 2003: Cyprus, England, Hong Kong SAR, Latvia-LSS, New Zealand, and Slovenia. At the same time, fourth-graders in Norway showed measurable declines in average mathematics and science achievement over the same time period.
  • U.S. fourth-grade girls showed no measurable change in their average performance in mathematics and science between 1995 and 2003 (figures A and B). U.S. fourth-grade boys also showed no measurable change in their average mathematics performance, but did show a measurable decline in science performance over the same time period.
  • U.S. Black fourth-graders improved in both mathematics and science between 1995 and 2003 (figures A and B). Hispanic fourth-graders showed no measurable changes in either subject, while White fourth-graders showed no measurable change in mathematics, but declined in science.
  • As a result of changes in the performance of Black and White fourth-graders, the gap in achievement between White and Black fourth-grade students in the United States narrowed between 1995 and 2003 in both mathematics and science (figures A and B). In addition, the gap in achievement between Black and Hispanic fourth-graders also narrowed in science over the same time period.
  • In 2003, U.S. fourth-graders in U.S. public schools with the highest poverty levels (75 percent or more of students eligible for free or reduced-price lunch) had lower average mathematics and science scores compared to their counterparts in public schools with lower poverty levels (figures A and B).
  • Eighth-graders in the five Asian countries that outperformed U.S. eighth-graders in mathematics in 2003-Chinese Taipei, Hong Kong SAR, Japan, Korea, and Singapore-also outperformed U.S. eighth-graders in science in 2003, with eighth-graders in Estonia and Hungary performing better than U.S. students in mathematics and science as well. Students in three of these Asian countries-Chinese Taipei, Japan, and Singapore-outperformed both U.S. fourth- and eighth-graders in mathematics and science on average.
  • U.S. eighth-graders improved their average mathematics and science performances in 2003 compared to 1995. The growth in achievement occurred primarily between 1995 and 1999 in mathematics, and between 1999 and 2003 in science. Moreover, the available data suggest that the performance of U.S. eighth-graders in both mathematics and science was higher in 2003 than it was in 1995 relative to the 21 other countries that participated in the studies (tables D and E).
  • In addition to students in the United States, eighth-graders in six other countries showed significant increases in both mathematics and science in 2003 compared to either 1999 or 1995: Hong Kong SAR, Israel, Korea, Latvia-LSS, Lithuania, and the Philippines. On the other hand, eighth-graders in nine countries declined in their mathematics and science performance in 2003 compared to either 1999 or 1995.
  • In 2003, U.S. eighth-grade boys and girls, and U.S. eighth-grade Blacks and Hispanics, improved their mathematics and science performances from 1995 (figures C and D). As a result, the gap in achievement between White and Black eighth-graders narrowed in both mathematics and science over this time period.
  • In 2003, U.S. eighth-graders in U.S. public schools with the highest poverty levels (75 percent or more of students eligible for free or reduced-price lunch) had lower average mathematics and science scores compared to their counterparts in public schools with lower poverty levels (figures C and D).


Table B. Average mathematics scale scores of fourth-grade students, by country: 1995 and 2003

Table B.  Average mathematics scale scores of fourth-grade students, by country: 1995 and 2003
1Met international guidelines for participation rates in 2003 only after replacement schools were included.

2Hong Kong is a Special Administrative Region (SAR) of the People's Republic of China.

3Designated LSS because only Latvian-speaking schools were included in 1995. For this analysis, only Latvian-speaking schools are included in the 2003 average.

4In 1995, Maori-speaking students did not participate. Estimates in this table are computed for students taught in English only, which represents between 98-99 percent of the student population in both years.

NOTE: Countries are ordered based on the average score. Parentheses indicate countries that did not meet international sampling or other guidelines in 1995. All countries met international sampling and other guidelines in 2003, except as noted. See NCES (1997) for details regarding 1995 data. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between the United States and one country may be significant while a large difference between the United States and another country may not be significant. Countries were required to sample students in the upper of the two grades that contained the most number of 9-year-olds. In the United States and most countries, this corresponds to grade 4. See table A1 in appendix A of the full report for details.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA),Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as table 6 on p. 8 of the complete report from which this article is excerpted.)


Table C. Average science scale scores of fourth-grade students, by country: 1995 and 2003

Table C.  Average science scale scores of fourth-grade students, by country: 1995 and 2003
1Met international guidelines for participation rates in 2003 only after replacement schools were included.

2Hong Kong is a Special Administrative Region (SAR) of the People's Republic of China.

3Designated LSS because only Latvian-speaking schools were included in 1995. For this analysis, only Latvian-speaking schools are included in the 2003 average.

4In 1995, Maori-speaking students did not participate. Estimates in this table are computed for students taught in English only, which represents between 98-99 percent of the student population in both years.

NOTE: Countries are ordered based on the average score. Parentheses indicate countries that did not meet international sampling or other guidelines in 1995. All countries met international sampling and other guidelines in 2003, except as noted. See NCES (1997) for details for 1995 data. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between the United States and one country may be significant while a large difference between the United States and another country may not be significant. Countries were required to sample students in the upper of the two grades that contained the most number of 9-year-olds. In the United States and most countries, this corresponds to grade 4. See table A1 in appendix A of the full report for details.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as table 12 on p. 18 of the complete report from which this article is excerpted.)

Back to top


References

Martin, M.O., Mullis, I.V.S., and Chrostowski, S.J. (Eds.). (2004). TIMSS 2003 Technical Report: Findings From IEA's Trends in International Mathematics and Science Study at the Eighth and Fourth Grades. Chestnut Hill, MA: Boston College.

Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., and Chrostowski, S.J. (2004). TIMSS 2003 International Science Report: Findings From IEA's Trends in International Mathematics and Science Study at the Eighth and Fourth Grades. Chestnut Hill, MA: Boston College.

Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., and Chrostowski, S.J. (2004). TIMSS 2003 International Mathematics Report: Findings From IEA's Trends in International Mathematics and Science Study at the Eighth and Fourth Grades. Chestnut Hill, MA: Boston College.

National Center for Education Statistics, U.S. Department of Education. (1997). Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics and Science Achievement in International Context (NCES 97-255). Washington, DC: U.S. Government Printing Office.


Footnotes

*Table A7 in appendix A of the full report groups the participating countries by continent and membership in the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of 30 industrialized countries that serves as a forum for members to cooperate in research and policy development on social and economic topics of common interest.


Data Source: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995, 1999, and 2003.

For technical information, see the complete report:

Gonzales, P., Guzmán, J.C., Partelow, L., Pahlke, E., Jocelyn, L., Kastberg, D., and Williams, T. (2004). Highlights From the Trends in International Mathematics and Science Study (TIMSS) 2003 (NCES 2005-005).

Author affiliations: P. Gonzales, NCES; J.C. Guzmán, L. Partelow, and E. Pahlke, Education Statistics Services Institute; L. Jocelyn, D. Kastberg, and T. Williams, Westat.

For questions about content, contact Patrick Gonzales (timss@ed.gov).

To obtain the complete report (NCES 2005-005), call the toll-free ED Pubs number (877-433-7827) or visit the NCES Electronic Catalog (http://nces.ed.gov/pubsearch).


back to top


Table D. Average mathematics scale scores of eighth-grade students, by country: 1995 and 2003

Table D.  Average mathematics scale scores of eighth-grade students, by country: 1995 and 2003
1Hong Kong is a Special Administrative Region (SAR) of the People's Republic of China.

2Met international guidelines for participation rates in 2003 only after replacement schools were included.

3Designated LSS because only Latvian-speaking schools were included in 1995. For this analysis, only Latvian-speaking schools are included in the 2003 average.

4National desired population does not cover all of the international desired population.

NOTE: Countries are ordered by average score. Parentheses indicate countries that did not meet international sampling or other guidelines in 1995 or 2003. See appendix A of the full report for details regarding 2003 data. See NCES (1997) for details regarding 1995 data. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between the United States and one country may be significant while a large difference between the United States and another country may not be significant. Countries were required to sample students in the upper of the two grades that contained the largest number of 13-year-olds. In the United States and most countries, this corresponds to grade 8. See table A1 in appendix A of the full report for details.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as table 7 on p. 9 of the complete report from which this article is excerpted.)


Table E. Average science scale scores of eighth-grade students, by country: 1995 and 2003

Table E.  Average science scale scores of eighth-grade students, by country: 1995 and 2003
1Hong Kong is a Special Administrative Region (SAR) of the People's Republic of China.

2Met international guidelines for participation rates in 2003 only after replacement schools were included.

3National desired population does not cover all of the international desired population.

4Designated LSS because only Latvian-speaking schools were included in 1995. For this analysis, only Latvian-speaking schools are included in the 2003 average.

NOTE: Countries are ordered by average score. Parentheses indicate countries that did not meet international sampling or other guidelines in 1995 or 2003. See appendix A of the full report for details regarding 2003 data. See NCES (1997) for details regarding 1995 data. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between the United States and one country may be significant while a large difference between the United States and another country may not be significant. Countries were required to sample students in the upper of the two grades that contained the largest number of 13-year-olds. In the United States and most countries, this corresponds to grade 8. See table A1 in appendix A of the full report for details.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as table 13 on p. 19 of the complete report from which this article is excerpted.)

Back to top


Figure A. Average mathematics scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and poverty level: 1995 and 2003

Figure A. Average mathematics scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and poverty level: 1995 and 2003
*p<.05, denotes a significant difference from 2003 average score.

NOTE: Reporting standards not met for Asian category in 1995 and American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander for both years. Racial categories exclude Hispanic origin. Other races/ethnicities are included in U.S. totals shown throughout the report. Analyses by poverty level are limited to students in public schools only. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between averages for one student group may be significant while a large difference for another student group may not be significant. The United States met international guidelines for participation rates in 2003 only after replacement schools were included. See appendix A of the full report for more information.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as figure 1 on p. 11 of the complete report from which this article is excerpted.)


Figure B. Average science scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and poverty level: 1995 and 2003

Figure B. Average science scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and poverty level: 1995 and 2003
*p<.05, denotes a significant difference from 2003 average score.

NOTE: Reporting standards not met for Asian category in 1995 and American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander for both years. Racial categories exclude Hispanic origin. Other races/ethnicities are included in U.S. totals shown throughout the report. Analyses by poverty level are limited to students in public schools only. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between averages for one student group may be significant while a large difference for another student group may not be significant. The United States met international guidelines for participation rates in 2003 only after replacement schools were included. See appendix A of the full report for more information.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as figure 3 on p. 21 of the complete report from which this article is excerpted.)

Back to top


Figure C. Average mathematics scale scores of U.S. eighth-grade students, by sex, race/ethnicity, and poverty level: 1995, 1999, and 2003

Figure C. Average mathematics scale scores of U.S. eighth-grade students, by  sex, race/ethnicity, and poverty level: 1995, 1999, and 2003
*p<.05, denotes a significant difference from 2003 average score.

NOTE: Reporting standards not met for Asian category in 1995 or 1999. Reporting standards not met for American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander in 1995, 1999, and 2003. Racial categories exclude Hispanic origin. Other races/ethnicities are included in U.S. totals shown throughout the report. Analyses by poverty level are limited to students in public schools only. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between averages for one student group may be significant while a large difference for another student group may not be significant. The United States met international guidelines for participation rate in 2003 only after replacement schools were included. See appendix A of the full report for more information.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as figure 2 on p. 13 of the complete report from which this article is excerpted.)

Figure D. Average science scale scores of U.S. eighth-grade students, by sex, race/ethnicity, and poverty level: 1995, 1999, and 2003

Figure D. Average science scale scores of U.S. eighth-grade students, by sex, race/ethnicity, and poverty level: 1995, 1999, and 2003
*p<.05, denotes a significant difference from 2003 average score.

NOTE: Reporting standards not met for Asian category in 1995 or 1999. Reporting standards not met for American Indian or Alaska Native and Native Hawaiian or Other Pacific Islander in 1995, 1999, and 2003. Racial categories exclude Hispanic origin. Other races/ethnicities are included in U.S. totals shown throughout the report. Analyses by poverty level are limited to students in public schools only. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between averages for one student group may be significant while a large difference for another student group may not be significant. The United States met international guidelines for participation rates in 2003 only after replacement schools were included. See appendix A of the full report for more information.

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science Study (TIMSS), 1995 and 2003. (Originally published as figure 4 on p. 23 of the complete report from which this article is excerpted.)

Back to top