II. International Comparisons
International Comparisons of Mathematics and Science
NCES uses a combination of international and U.S. databases to look at the performance of our students. The combination of both types of data is required to see ourselves in stereographic or parallel perspective. U.S.-only data is blind in one eye, and international data is blind in the other. Both types of data are necessary for a clear and an accurate view of our students' performance.
TIMSS is noteworthy not only because of its scope and magnitude, but also because of innovations in its design. In this international study, NCES along with the National Science Foundation (NSF) combined multiple methodologies to create an information base that goes beyond simple student test score comparisons to examine the fundamental elements of schooling. Innovative research techniques include analyses of textbooks and curricula, videotapes, and ethnographic case studies. The result is a more complete portrait of how U.S. mathematics and science education differs from that of other nations, especially in extended comparisons with Germany and Japan.
The information in these reports can serve as a starting point for our efforts to define a "world-class" education. If the United States is to improve the mathematics and science education of its students, we must carefully examine not just how other countries rank, but also how their policies and practices help students achieve. TIMSS shows us where U.S. education stands -- not just in terms of test scores, but also what is included in textbooks, taught in the schools, and learned by students. Examining these data provides a valuable opportunity to shed new light on education in the United States through the prism of other countries. At the same time, we should avoid the temptation to zero in on any one finding or leap to a conclusion without carefully considering the broader context.
Our students' international standing declines as students progress through school, according to TIMSS. Overall, U.S. fourth-graders scored above the international average in both science and mathematics. Our eighth-graders scored above the international average in science but below it in mathematics. In twelfth-grade, the scores of both our overall student population tested on general mathematics and science knowledge, and of our more advanced students tested in mathematics and physics, were well below the international average.
Fourth-Grade Findings. In both mathematics and science, U.S. fourth-graders performed above the international average. In mathematics, of the 26 participating TIMSS countries, U.S. fourth-graders outperformed students in 12 countries and were outperformed by students in seven countries. In science, U.S. students outperformed students in 19 countries, and were outperformed by students in only one country-Korea. In the six mathematics content areas, U.S. fourth-graders exceeded the international average in five. In the science content areas, U.S. fourth-graders exceeded the international average in all four areas assessed.
Eighth-Grade Findings. Data on eighth-grade performance from TIMSS suggests a general improvement in U.S. eighth-grade science scores as compared to a prior 1991 international assessment that placed U.S. students below average, though the tests and the set of participating nations have changed. The TIMSS data, however, show that U.S. eighth-grade students' mathematics performance remains slightly below the international average. U.S. eighth-grade students scored lower, on average, in mathematics than students in Canada, France, and Japan, and scored about the same as students in England and Germany. In science, eighth grade students from the United States scored higher, on average, than students in France, about the same as students in Canada, England, and Germany, and lower than students in Japan. (Figure G) summarizes U.S. performance by content area on the fourth- and eighth-grade assessments.
Twelfth-Grade Findings. The twelfth-grade TIMSS included 21 countries that conducted assessments of their students' general knowledge in mathematics and science during their last year in secondary school. Japan and other Asian countries that traditionally perform well in mathematics and science did not participate in the twelfth-grade TIMSS. Even with those Asian countries excluded, the United States performed relatively poorly. In the mathematics general knowledge assessment, U.S. twelfth-grade students were outperformed by 14 countries, and outperformed two countries. U.S. students performed the same as students in four other countries. In science, U.S. twelfth-grade students were outperformed by students in 11 countries, and outperformed students in two countries . U.S. students performed the same as students in seven other countries (Figure H).
Average test scores can mask important differences in the distribution of scores. For example, as a result of our country's diverse population, U.S. test score averages could be unduly lowered by a relatively large group of low-scoring students. In the twelfth-grade TIMSS assessments, however, the distribution of scores among U.S. students was no wider than that in most other participating countries; the U.S. scores also start and end lower than those in higher scoring countries. We also like to think that at least America's "best and brightest" students are among the smartest in the world; again, TIMSS findings suggest otherwise. Sixteen countries assessed advanced mathematics and physics among a select group of advanced students. In advanced mathematics, 11 countries outperformed the U.S., and no countries performed more poorly. In physics, 14 countries outperformed the U.S.; again, no countries performed more poorly (Figure I).
Several other factors suggested by observers also do not account for the relatively poor performance of U.S. students in grade 12. For example, it is not the case that a greater proportion of U.S. students complete secondary school than in most of the other countries participating in this phase of TIMSS. Thus, the vast majority of U.S. young people are not being compared only to an elite in other countries. Furthermore, in TIMSS, the general pattern was that countries with higher proportions of young people enrolled in and completing secondary school outperformed countries with lower proportions. The decentralized nature of decision-making about curriculum did not explain the poor performance of U.S. students. Some countries with decentralized decision-making outperformed us and some did not. The same was true of countries with centralized decision-making. Finally, while U.S. students on average were about a half a year younger than the average for all 21 counties, the age differential is not a major factor contributing to our poor performance. Not only is the age differential relatively small (and it is even less in the advanced assessments), countries in which the average age of the students was similar to or younger than the U.S. also outperformed us.
Among the other achievement findings drawn from the TIMSS:
- In comparison with their international counterparts, U.S. students performed better in science than in mathematics at all three grade levels;
- Among U.S. students, there is no significant gender gap in mathematics at any grade level for the general populations tested. However, in fourth-grade and twelfth-grade science, and in twelfth-grade advanced mathematics and physics, male students performed better than female students.
- At grade 4, 16 percent of U.S. students would be in the international top 10 percent in science; at grade 8, 13 percent;
- In mathematics, 9 percent of U.S. fourth-graders would be in the international top 10 percent in mathematics, compared to 5 percent of eighth-graders.
Lessons From TIMSS
While TIMSS has given us information on our international standing, it is most valuable in telling us what factors are related to high achievement in schools. The overarching message is that there is no easy solution or single nostrum that will magically increase our nation's performance. Indeed, TIMSS shows us that many of the cure-alls recommended in the past are not associated with high performance in all nations. For example, more seat time in math and science, more homework, and less television have often been recommended as methods for increasing student performance.
These strategies may indeed be effective in the case of individual students or schools, yet TIMSS has shown us another perspective. Comparisons of eighth-grade students, teachers, and classrooms in the U.S., Japan, and Germany have been particularly revealing. For example, U.S. eighth graders already spend more seat time in math and science classes than students in Japan and Germany. Japan outperforms us at this grade level, while Germany does not, so this shows that more seat time is not necessarily a magic tonic. With respect to homework, U.S. eighth-grade teachers already assign more homework, spend more class time discussing it, and are more likely to count it toward grades than teachers in Japan. Japanese eighth graders also watch just as much TV as students in the U.S. The most recent TIMSS also found that the relatively poor performance of U.S. twelfth-grade students is not related to hours spent on homework, the use of calculators or computers, time spent watching television or working at a paid job, or to attitudes toward mathematics and science.
These and other TIMSS findings show us that there is no single easy answer to achieving high performance in mathematics and science. But the TIMSS and other NCES data sources do suggest some problems in U.S. mathematics and science education that may help explain our relatively low achievement at the higher grade levels. These data suggest that three issues are worth our attention: curriculum, coursetaking, and teacher preparation.
First, both the mathematics and science curricula in American high schools have been criticized for their lack of coherence, depth, and continuity-for covering too many topics at the expense of in-depth understanding. As a result, our secondary school curricula leave American students with a more limited opportunity to learn than their counterparts have in other countries. For example, while most other countries introduce algebra and geometry in the middle grades, in the U.S. only 25 percent of students take algebra before high school. The TIMSS also demonstrated the relative "slowness" of our curricula. The study found that the topics on the twelfth- grade general knowledge mathematics assessment were covered by the ninth grade in the U.S, but by the seventh grade in most other countries. The topics on the general science assessment were covered by the eleventh grade in the U.S., but by the ninth grade in most other countries.
Students' exposure to challenging mathematics and science content is further limited by their coursetaking behavior. Despite some recent increases in academic coursetaking, fully 90 percent of all U.S. high school students stop taking mathematics before getting to calculus. Even among college-bound seniors, 52 percent have not taken physics, 48 percent have not taken trigonometry, and 77 percent have not taken calculus; almost one-third (31 percent) had not taken four years of mathematics. Among 1994 high school graduates, only 9 percent had taken calculus and 24 percent had taken physics.
Finally, courses and curricula do not teach themselves. At the most basic level, the education system relies on knowledgeable, well-trained teachers to convey the information students need to learn. What teachers do not know, they cannot teach. And our data suggest that considerable percentages of our mathematics and science teachers have not been adequately exposed to the information they teach. (Figure J) shows that in 1993-94, 28 percent of public high school (grade 9-12) mathematics teachers and 18 percent of public high school science teachers were teaching out-of-field (that is, without a major or minor in their subject). Within science sub-fields, 31 percent of life science (biological/life sciences) teachers and 55 percent of physical science (chemistry, physics, earth science, and physical science) teachers lacked a major or minor in their sub-field. In addition, 24 percent of mathematics teachers and 17 percent of science teachers lacked state certification in their teaching field.
In short, TIMSS does dispel myths, but more importantly, it shows us our own education system in clearer perspective. In our quest for factors related to better student performance, TIMSS encourages us to focus on rigorous content, focused curriculum, good teaching, and good training for teachers. TIMSS has shown us that the typical U.S. eighth-grade mathematics class usually discusses material taught at the seventh-grade around the world. Compared to those in Japan, our mathematics teachers tend to focus on teaching specific math skills, rather than higher-level mathematical problem solving. For example, U.S. eighth-grade math teachers are more likely to merely state rather than explain mathematical concepts. Further, our curriculum includes more topics, and our teachers are more frequently interrupted by loudspeakers and other outside agents, while they are teaching than are teachers in Japan and Germany. Our teachers also lack a one or two year apprenticeship in teaching before they become teachers, as is the case in these two other countries. Clearly TIMSS shows us that while it may not be easy, important change is needed to help our nation continue to improve its performance.
International Comparisons of Reading
In 1991, the IEA Reading Literacy Study assessed the reading literacy of fourth-graders (in 27 countries) and ninth-graders (in 31 countries). The underlying framework for this assessment paralleled the NAEP framework in that it too defined reading in terms of three text types - narrative, expository and document. In contrast to the NAEP Reading Report Card, this study painted a more positive picture of the reading literacy of American students.
- American fourth-graders were outperformed only by Finland; U.S. students performed about the same as students from Sweden, while outperforming students from 24 other nations.
- American ninth-graders' performance was equivalent to that of students from 15 other nations; Americans outperformed students from 14 nations, while only the students from Finland did better than our students.
Considering only those countries that were then part of the Organization for Economic Cooperation and Development (OECD), the study's findings indicate that:
- Among fourth-graders the reading performance of about 60 percent of U.S. students meets or exceeds the OECD average on two scales - narrative (which corresponds roughly with the NAEP "reading for a literary experience" scale), and expository (which corresponds roughly with the NAEP "reading to be informed" scale). About 70 percent of American fourth graders meet or exceed the OECD average on the third IEA reading scale - documents (which roughly corresponds with the NAEP "reading to perform a task" scale).
- The comparative advantage of American ninth-grade students was not as great as that of the fourth graders. Between 52 and 55 percent of U.S. ninth-graders met or exceeded the OECD average on the three scales.
- Most groups of American students, even the most disadvantaged, outperform the OECD reading average, with only a few exceptions - black students in both grades and students in 9th grade whose parents did not complete high school do not consistently meet or exceed the OECD average.
The difference between the NAEP view of America's fourth, eighth, and twelfth-grade students' reading proficiency and that emerging from the IEA data may be attributed to two very important differences in these assessments. First, there are distinct differences in the way that the data are benchmarked. IEA reporting is based on comparisons of student performance across countries, while much of NAEP reporting is based on student performance against a desired standard defined by NAGB. Second, the IEA test mainly asks students to recognize details and to make simple inferences and literal interpretations while the NAEP test goes further, i.e., requiring students to identify themes to detect the author's point of view, to make larger inferences, and to state a position with supporting citations from the text. These differences in benchmarking and in test rigor raise important questions. Primarily, we must consider what benchmarks are reasonable for our society. One way to examine this issue is to look at achievement or proficiency data in relation to important outcome measures, as we will discuss next.
International Perspective on Labor Force Proficiency
Literacy has been viewed as one of the fundamental tools necessary for successful economic performance in industrialized societies. As society becomes more complex and low-skill jobs continue to disappear, concern about adults' ability to use written information to function in society continues to increase. Within countries, literacy levels are affected both by the quality and quantity of the population's formal education, as well as by participation in informal learning activities.
The most recent international adult literacy data (1996) demonstrate that the U.S. appears most similar to New Zealand and the United Kingdom in the overall distribution of literacy skills. (Figure K) These three countries had close to 20 percent of their adult population at both the high and low ends of the literacy scale (Level 1 and Levels 4 and 5). In contrast, the performance of our European counterparts was concentrated in the middle literacy levels, with at least two-thirds of the adult population in the Netherlands, Switzerland (both French and German speaking) and Germany at Literacy Levels 2 or 3. While Sweden tended to have the greatest concentration at the higher end of the scale, Poland's adults were concentrated at the lower end.
In the United States, as you might expect, workers with higher adult literacy scores are unemployed less and earn more than workers with lower literacy scores. Unemployment rates are especially high for workers in the two lowest levels of literacy-levels 1 and 2-on each of the three literacy scales. For these workers, the unemployment rate ranges from 12 percent for workers with level 2 quantitative literacy to nearly 20 percent for those with level 1. Unemployment rates for individuals in the two highest literacy levels-levels 4 and 5-are less than 6 percent.
Workers with high literacy scores earn more than other workers do, on average (Figure L). On the prose scale, for example, full-time workers in level 3 earn a mean weekly wage 50 percent higher than that of their counterparts in level 1. Those in level 5 earn a weekly wage 71 percent higher than the wage of those in level 3. Thus, academic skills do make a difference in both earnings and employability.
III. Changes in Student Behavior Since A National at Risk
In addition to reviewing changes in student achievement since A Nation at Risk , as well as our comparative international educational standing, it is instructive to look at other significant changes in the educational landscape since the publication of this seminal work. Three are especially worthy of note: the decline in the high school dropout rate, an increase in the educational aspirations and college attendance rates of high school seniors, and increases in the academic course load of high school students. Each of these changes indicate noteworthy positive responses to what was called for in A Nation at Risk.
Dropping Out of School
There has been a reduction in the drop out rate since A Nation at Risk. Most of this decline occurred during the 1980s, and was especially pronounced for blacks. Over the last decade, 300,000 to 500,000 students in grades 10 through 12 left school each year without successfully completing a high school program. In October 1996, nearly 3.6 million 16- to 24-year-old youth were not enrolled in a high school program and lacked a high school credential. These young adults accounted for 11 percent of the 32 million 16- to 24-year-olds in the United States. Nevertheless, this 1996 dropout rate was three points lower than the 1982 dropout rate of 14 percent. And, the dropout rate for Black youth during this period fell from 18 to 13 percent.
The dropout rates of 16-to-24-year-olds Hispanics remained at levels substantially higher than the dropout rates experienced by their white and black peers (Figure M). And, in contrast to the decline among black and white 16-to-24-year olds, the dropout rates for Hispanics has not changed significantly since 1982. In 1996, 29 percent of Hispanics were not enrolled in school and had not completed high school; however this percentage includes young immigrants who came to the United States without high school credentials and never enrolled in a U.S. school . The dropout rate for Hispanic immigrants aged 16- to 24-years-old was 44 percent, compared to the dropout rate for first-generation Hispanics born in the United States, which was 17 percent (Figure N).
Educational Aspirations and College Attendance
One of the most dramatic changes taking place since A Nation at Risk is that the hopes of high school seniors for the future increasingly include more education. In 1992, 69 percent of seniors said that they hoped to graduate from college, compared with 39 percent of 1982 seniors. Moreover, 33 percent said they hoped to earn a postgraduate degree as compared with 18 percent in 1982. The proportion of minority students aspiring to postgraduate degrees was about the same, or higher, than for whites. Not surprisingly, these higher student aspirations have been accompanied by substantial increases in actual college attendance. The proportion of high school graduates going directly on to college rose from 51 percent in 1982 to 65 percent in 1996.
Coursetaking Patterns in High School
One of the important elements in the recommendations in A Nation at Risk was to increase the academic course load of high school students. Since the release of that report, most states have raised course requirements for high school graduation and most states have mandated student-testing standards. As a result, both college-bound and non-college-bound students now take more academic courses than their counterparts did a decade before. In 1982, the average high school graduate completed 2.6 Carnegie units in mathematics and 2.2 units in science. By 1994, the average number of Carnegie units completed had risen to 3.4 in mathematics, and 3.0 units in science. Foreign language units rose from 1.0 to 1.8, and coursework in English and social studies also increased. The increase in the average units completed means that more students are now taking advanced mathematics courses, such as calculus, which was completed by 9 percent of the 1994 graduates compared to 5 percent of the 1982 graduates. Similarly, the proportion of graduates completing a physics course rose from 14 percent in 1982 to 24 percent in 1994 (Figure O).
A Nation at Risk recommended that high school students complete a "New Basics" curriculum that included a minimum number of courses in the core academic areas of English (4), Mathematics (3), Science (3), and Social Studies (3). Since the release of these "New Basics" recommendations, high school graduates have taken more courses overall, particularly academic courses. The proportion of students completing the "New Basics" core curriculum in English, mathematics, science, and social studies has increased; and greater percentages are taking Advanced Placement (AP) courses. In 1982, 14 percent of high school graduates earned the credits recommended in A Nation at Risk; by 1994, 50 percent had done so. The percentage of graduates who have completed the more extensive recommendations for college-bound students, which include the "New Basics," plus 2 years of foreign language instruction and a half-year of computer science, rose from 2 percent in 1982 to 25 percent in 1994.
Even though we cannot establish a cause and effect relationship, it is interesting to compare the average mathematics and science performance of 17-year-olds, as measured by our National Assessment of Education Progress, and the increase in course taking. The mathematics performance of 17-year-olds rose by 7 points between 1982 and 1994, which roughly equates to about 2/3 of the typical grade to grade progress. This increase compares closely to the rise of .8 average units of mathematics completed by high school graduates. The science performance of 17-year-olds rose by 11 points between 1982 and 1994, compared to an average increase of .9 science units completed by high school graduates.
Conclusion
Whatever else one might argue is the legacy of A Nation at Risk, it clearly signaled the recognition of educational performance as a national concern, an issue of national importance. In times like this, Federal statistical agencies, such as the National Center for Education Statistics, play a critical role.
First, they provide the data that researchers and statistical analysts need. As demonstrated by the numbers presented here, there are large differences in how well students do -- across time, across countries, and sometimes across groups. It falls typically to researchers to untangle these relationships, to separate educational inputs from outputs, and to identify the processes that contribute most powerfully to student performance. Second, statistical agencies aid policymakers in a more direct manner -- by sounding alarms when problems arise or issues emerge that deserve public attention. Informing policymakers with data that clarify where problems exist and what issues are most pressing is one of the Federal government's most vital roles.