Skip Navigation

Education Statistics and Education Policy: The American Experience

Pascal D. Forgione, Jr., Ph.D.
Commissioner of Education Statistics
National Center for Education Statistics
U.S. Department of Education
1990 K Street NW
Washington, DC 20006

Delivered at:
Institute of European and American Studies
Academia Sinica
Taipei, Taiwan, R.O.C.
November 12, 1998

Introduction
Good afternoon everyone. I am pleased to have this opportunity to speak here at the Institute of European and American Studies regarding my agency, the National Center for Education Statistics (NCES) of the United States, and about possibilities and benefits of Taiwan’s increased participation in international education research. It was not too long ago in the United States that international comparisons of education systems were conducted in an atmosphere of competitiveness and rivalry. But over the past decade we have witnessed a gradual transition towards examining other country experiences and using international research to improve our schools. As more countries take part in international research projects, and as we explore together the similarities and differences between systems in greater depth, our joint efforts become truly an international cooperative undertaking. It is in this spirit that I present my remarks today.

Education policy making in the United States is becoming more data-driven. At the federal, state and local levels, there is a larger recognition of the importance of interesting, compelling information. In this new global economy and information age, data is being used, analyzed, communicated, and debated more than ever before. Dominating the public’s attention in primetime and newspaper headlines have been storie3s about serious, even critical educational issues backed by solid data, such as overcrowded schools, school crime, and the quality (or lack thereof) in the teaching profession. Recent television productions on equity and education in the United States featured education leaders using data forcefully to argue their points. Stories on education issues that were once based on anecdotes and opinions are being replaced by the explosion of the use of solid information and data sources. Though this trend is exciting for those of us in the data and analysis business, it is also sobering to those of us at NCES. We have a large job to do, and it is getting larger.

I. Background on NCES
Many of you may already be aware of some of the activities and publications of my agency, which I will refer to from here on as "NCES." Before I present a detailed description of NCES, I would like to describe briefly how NCES fits in the federal government and in the overall picture of education policy and research in the United States.

As the name implies, NCES is a part of the federal government. It lies within the United States Department of Education, which is headed by Secretary of Education, a member of the President’s Cabinet. Within the Department, NCES is part of the Office of Educational Research and Improvement, or "OERI," and is headed by an Assistant Secretary. The position of NCES Commissioner, in which I currently serve, is a position filled by a Presidential appointment who must be confirmed by the United States Senate. The Commissioner serves for a four-year term.

NCES is one of 13 federal statistical agencies, and the primary one for collecting and analyzing data related to education. Although the structure of the federal agencies has changed throughout our country’s history, NCES has been in existence in one form or another since 1867. Though the federal government remains the junior government partner in education, working with state and local education agencies, statistics is one area that has always been considered central to the federal role. Our current mission, as mandated by Congress, is to do the following:

  • Collect, collate, analyze, and report full and complete statistics on the condition of education in the United States and other Nations;
  • Conduct and publish reports and specialized analyses of the meaning of such statistics; and
  • Assist state and local education agencies in improving their statistical systems.

The most common end results of these activities, and certainly the most visible ones, are our publications. NCES issues nearly 200 publications each year. These documents include statistical reports, bulletins, early releases of preliminary findings, directories, and handbooks of standard terminology. Many of these publications report the findings of specific surveys, but at least three— The Condition of Education, the Digest of Education Statistics, and the Projections of Education Statistics, —cover the field of education statistics from a broad perspective. I will discuss our publications in more detail in the next section of my talk.

In summary, NCES has been well managed with a clear vision for the future. NCES had acquired a reputation for trustworthy, useful data and long-range planning that has given the organization great stability and quality, even under adverse circumstances. As Commissioner, I have established four values to guide NCES’ national data agenda, namely: regularity, quality, usefulness and timeliness. Consequently, NCES has a strong foundation for quality data collection and use. Consequently, NCES shares with other statistical agencies, however, in the realities of having to do more with less. That has been the pattern for the last few years and will intensify in the near future. In FY99, NCES staff consists of 116 positions, some 15% less than a decade ago. Yet, since FY85, there has been an unprecedented growth of NCES mission and responsibility, from $12.7 million to $104 million in FY99 (or, 818% increase).

The Office of the Commissioner sets policy and standards for NCES and provides oversight for the operation of it, ensuring that statistical quality and confidentiality are maintained. Two standing bodies of individuals external to NCES provide input to the Office of the Commissioner. The Advisory Council on Education Statistics (ACES) provides general input regarding the activities of NCES, while the National Assessment Governing Board (NAGB) provides oversight specifically for the National Assessments of Educational Progress.

The Statistical Standards and Methodology Group provides state-of-the-art technical and statistical support to NCES and to federal and nonfederal organizations and entities involved in statistical work in support of it. In addition, the staff develops and produces the book, Projections of Education Statistics, and operates the system of licensing for individuals and organizations that require access to confidential data for statistical purposes.

The Data Development and Longitudinal Studies Group is responsible for focusing the content and design of the National Center for Education Statistics programs; improving analyses, distribution, and use of products; and increasing international activity. The staff also produces the major flagship publications of the center: the Digest of Education Statistics, and The Condition of Education.

The Surveys and Cooperative Systems Group oversees planning, design, operations, statistical analysis, reporting and dissemination for elementary, secondary, vocational, postsecondary, and library surveys. It ensures that quality and confidentiality are maintained. The staff is also responsible for the development and implementation of the Congressionally mandated National Cooperative Education Statistics System in areas of elementary and secondary education, postsecondary education, and library statistics.

Finally, the Assessment Group creates, designs, develops, implements and reports on the National Assessment of Educational Progress at the national level and coordinates assessment and related data collection activities with the states. The staff also conducts a variety of other related education assessment studies.

As a federal agency, we must always bear in mind that our role is to serve: we fulfill the needs of our constituency, or our "customers" as we like to call them. Gradually over the years, the makeup of our customers has changed. Certainly policy makers have always comprised a key part, as have educational researchers. Recently, however, we have been expanding our range of customers. We feel strongly that the data we collect and analyze can be useful not only to national- and state-level policy makers and researchers, but also to all teachers, parents, school boards, school district superintendents, school principals, and anyone else involved in the reform and improvement of learning. Thus, although we serve by fulfilling the needs and requests of our traditional customers, we also serve by actively reaching out to new and different customers who we feel can be assisted by our information. As a result, we are a very public and visible agency, perhaps the most so of any of the 13 federal statistical agencies.

II. Programs and Plans of NCES
In talking about the activities of NCES, it is perhaps easiest to talk in terms of the type of questions we answer, the ways in which we answer them, and how these answers are reported.

Administrative Data on Students, Teachers, and Schools and Sample Surveys

First, if we think about the most basic questions about an education system, they might include student enrollments, numbers of schools and teachers, and expenditures on education. In the United States, we would like to be able to analyze this information in several different ways, including by inter-state comparisons and trend data from previous years. For this type of information, NCES has several ongoing data collection activities. These activities survey all organizations and institutions within their categories. In other words, they are universe surveys.

Our primary database for basic information on public elementary and secondary education is the Common Core of Data. The Common Core of Data is based on an annual survey of all public elementary and secondary schools, and all school districts in the United States. It provides general descriptive statistics about some 85,000 public schools and some 14,000 school districts, demographic information about students and staff, and fiscal data.

The same type of data on private schools is collected by the Private School Survey, which is conducted every two years. In addition to the type of data found in the Common Core of Data, the Private School Survey includes such variables such as religious or other affiliation and program emphasis, for some 25,000 private schools across the U.S.

Basic information regarding postsecondary education can be found in the Integrated Postsecondary Education Data System (IPEDS). This database, which is updated annually, includes information on fall enrollment, number of degrees offered, faculty salaries, financial statistics, and library statistics. All of these data exist at the institution level for some 10,000 higher education and post-secondary institutions, including some 3,500 two and four year colleges and universities.

If we think further about needs for educational data, we would probably be interested in background characteristics of students, schools, and teachers, such as home and school environments. Because this information is considerably more complex than basic data on schools, it would unfeasible to survey all members of the group. So, instead, we use sample surveys or cross-sectional studies.

The Schools and Staffing Survey (SASS) surveys state education agencies to produce national and state-level data on public and private schools, principals, school districts, and teachers. It monitors supply and demand conditions, teacher work force composition, school policies, and the general status of teaching and schooling. It facilitates comparison between public and private schools and allows linkages of data on teachers, schools, school districts, and administrators.

The National Study of Postsecondary Faculty (NSOPF) examines faculty and staff characteristics, including socio-demographic characteristics, field of instruction, professional background, courses taught, and tenure. Two cycles of NSOPF have been conducted to date, allowing for comparisons to be made over time and detailed comparisons among faculty in various disciplines.

The National Postsecondary Student Aid Study (NPSAS) reports on the costs of higher education, distribution of financial aid, and characteristics of aided and non-aided students. Administrative records concerning student financial aid are coupled with student interviews and data from a sub-sample of parents.

Because the home environment is so crucial to a student’s success in school, NCES collects information on the home environment through the National Household Education Survey (NHES). It is a telephone survey of households that covers varied educational topics, such as program participation, home activities, early childhood and adult education, as well as parental involvement in education, and the role of families in children’s learning. It also includes extensive family and household background information as well as characteristics of the school attended by the child. The survey addresses high priority topics on a regular basis, but also includes one-time surveys on topics of particular interest.

Altogether, the databases and surveys I just mentioned provide an extremely rich source of information on virtually any non-achievement related aspect of our education system. We have data on the full range of public and private schools, teachers, and students, beginning with pre-kindergarten programs, continuing through primary, secondary, postsecondary, and adult education programs. Data exist at the national, state, local, and, in some cases, the institution level. We have basic data on schools and students, but also detailed information on student and teacher backgrounds, and home and school environment. Data can be analyzed using a variety of variables, including socioeconomic ones such as population of the local community, ethnicity and poverty status of students and school populations, education level of parents, and languages spoken at home. Because most of these surveys are conducted on a regular basis, it is possible to examine trends from year to year.

As just hints of the potential of these databases, we can answer such basic and detailed questions as:

  • How many schools and universities are there in the United States, how many students do they enroll, and how many teachers do they employ?
  • How have public and private school and university enrollments changed over time?
  • Are minority students and students from low-income families participating in early childhood education programs at the same rate as non-minority students and students from and middle- and high-income families?
  • How well do public and private school teachers feel they are involved in making key policy decision?
  • How much time do teachers spend on school-related activities before and after school and on weekends?
  • What percentage of postsecondary students receives financial aid, what are the types and sources of that aid, and what are the average amounts? What are background characteristics of students receiving aid versus those not receiving aid?
  • What percentage of university students’ classroom hours is spent with senior faculty members?
  • How has the number of bachelor’s degree awarded in computer and information science changed over time?
  • Is parents’ choice of residence influenced by where their child would go to school?

Longitudinal Studies

All of the surveys and studies I mentioned so far are updated on a regular basis, many of them every year. Thus, they can show trends over time. However, for more detailed information on what happens to particular groups of students as they grow older, NCES conducts several longitudinal studies. These studies follow cohorts of students of varying initial ages, provide detailed information on academic growth, high school dropouts, and experiences in postsecondary education and the world of work. These studies are long-term surveys that follow specific classes of students before, during, and after both high school and college. At the end of each study, educational and occupational aspirations are compared against actual attainment. Our longitudinal studies include:

  • The National Longitudinal Study of the High School Class of 1972 (NLS72), which followed a sample of 1972 high school senior seniors from 1972 to 1986.
  • High School and Beyond (HS&B), which followed 1980 tenth-grade students from 1980 through 1992, and 1980 twelfth-grade students from 1980 through 1986.
  • The National Educational Longitudinal Study of 1988 (NELS:88), which surveyed eighth-grade students in 1988 and will continue surveying them through the year 2000.
  • The Early Childhood Longitudinal Study, which began this year by surveying a sample of kindergarten children and will survey them through the fifth grade.

In addition, NCES is conducting two postsecondary longitudinal studies:

  • The Beginning Postsecondary Student (BPS) longitudinal study is a survey of all students, regardless of age, who began postsecondary education in the school years 1989-90 and 1995-96. It describes their experience during higher education and future work status, as well as family formation.
  • The Baccalaureate and Beyond (B&B) study surveys a sample of 1992-93 baccalaureate degree completers one year after graduation. It examines occupational, educational, and family, experiences of college graduates over time. The study will follow each cohort to gather information concerning delayed entry into graduate education, times to completion of graduate education, and the interactions between work and education at the graduate level.

What makes these longitudinal studies so powerful is their ability to draw profiles of groups of students and follow them as they move from childhood, through adolescence, and into adulthood. We are able to see, for example, the educational paths students take, including such key decisions as choices of programs, courses, and majors and the decision to enroll in postsecondary education. We see where students with different background characteristics encounter roadblocks in their education and how successful they are in persisting. And we can see what students with different cultural, family, and educational backgrounds choose to do after high school and can follow their progress in postsecondary education and the work force. As I will discuss later, these type of studies are extremely useful in studying the effects of particular educational and social policies and in designing new ones.

Assessments

One of the most important—and certainly the most closely watched—aspects of any education system is academic achievement. For many people, the bottom line in education is whether students are acquiring the skills and knowledge taught in their schools.

To answer this question on a national level, NCES conducts the National Assessment of Educational Progress (NAEP). NAEP is mandated by Congress and was first administered in 1969. It assesses the knowledge and skills of the nation’s youth in reading, mathematics, science, writing, history, geography, and other subject areas. It involves representative samples of fourth-, eighth-, and twelfth-grade students. In addition to the national-level results, NAEP also provides comparative state data for a large number of states that voluntarily participate in selected assessments. NCES is responsible for collecting and reporting data on a periodic basis, and ensuring valid and reliable trend reporting. The accompanying charts overview the history of NAEP assessments and the recently adopted future NAEP assessment schedule.

The results of NAEP are reported as scores from 0 to 500. Over the past several years, NCES has been using a set of performance levels to report the NAEP results. The purpose of these proficiency levels is to establish objective criteria for different ranges of scores in terms of what students should know and be able to do. The advantage of these proficiency levels is that they measure student performance against a standard of what their performance should be, rather than simply whether their scores are higher or lower than students from previous years or in other states. The performance levels for the NAEP tests are developed by NAGB.

NCES also conducts the National Adult Literacy Survey (NALS), which measures the various literacy skills of the adult population of the United States. The next NALS will be collected in 2002 which is a decade since the original study.

International Activities

In addition to its national data collection activities, NCES is an active participant in several international data collection projects, many of which you are probably familiar with. Over the past ten years, the Organization for Economic Cooperation and Development (OECD) has been conducting the International Education Indicators Project. The purpose of the project is to produce comparable data from OECD nations on education indicators of international importance. These include data on enrollments, educational attainment of the population, labor market outcomes, expenditure, and achievement. NCES staff members serve on the various

CHART 1

Historic NAEP Assessments

 

CHART 2

Future NAEP Assessments

advisory boards that help guide the project. We are also responsible for providing data on the United States. The indicators are contained in the report, Education at a Glance, and the basic data are also available on diskette.

NCES was also an active participant in the Third International Mathematics and Science Study (TIMSS) which is sponsored by the International Association for the Evaluation of Educational Achievement (IEA). We helped fund the International Study Center at Boston College/Boston Massachusetts/USA, which was responsible for carrying out the assessments of achievement for three populations (i.e., age 9, age 13, and end of secondary school). In addition, we sponsored several other research projects to complement the assessments of achievement. These included an analysis of textbooks and curriculum frameworks, a videotape study of age 13 or eighth-grade mathematics lessons in Germany, Japan, and the United States, and an in-depth case study of schools, students, and teachers in those same three countries for all three populations.

We are now taking a similarly active role in TIMSS-R, a follow-up to TIMSS to be conducted in 1999, in cooperation with IEA. TIMSS-R will administer mathematics and science examinations again to age 13 or eighth-grade students in approximately 40 countries, including Taiwan, many of which participated in the original TIMSS. The fact that age 13 or eighth-grade students in 1999 were age 9 or fourth grade students in 1995, the year the TIMSS assessments were originally administered, means that countries participating in both TIMSS and TIMSS-R will be able to identify four-year trends in science and mathematics achievement for this.

 

We are also in the midst of the launch of the Program of International Student Achievement (PISA 2000), the first cycle of a strategy of the OECD to produce indicators on student achievement of age 15 year old students in reading, mathematics, and science within its member countries. These efforts will result in international student data that address both subject-specific knowledge and broader cross-curricular elements and that are reported on a regular basis and in a timely and efficient manner.

To make this international data even more useful for the U.S. audience, NCES has developed linkages between international indicators and similar ones used in the United States at the state level. For example, a statistical link was developed between NAEP and TIMSS for Grade 8 math and science so that some 40 states received a projected TIMSS score based on the 1996 NAEP performances.

Publications

Almost all of the surveys and activities I have mentioned have associated with them publications reporting their results and methodologies. These publications can range from one-page issue briefs to multi-volume reports.

In addition to the publications specific to individual surveys or activities, NCES publishes three flagship reports that take a comprehensive look at all aspects of education.

The Condition of Education is an annual report to Congress that focuses on some 60 indicators, representing a consensus of professional judgment on the most significant national measures of the condition and progress of education. Indicators are grouped according to the following topics:

  • Access, participation, and progress;
  • Achievement, attainment, and curriculum;
  • Economic and other outcomes of education;
  • Size growth, and output of educational institutions;
  • Climate, classrooms, and diversity in educational institutions; and
  • Human and financial resources of educational institutions.

For each indicator, the book highlights noticeable trends, provides background information and advice for interpreting the indicator, and supplemental tables and full references.

The Digest of Education Statistics is an annual compendium of educational data, drawing from across all NCES activity areas and from outside sources as well. It addresses the areas of enrollment, teachers, educational outcomes, and finance.

Projections of Education Statistics is an annual report that provides projections for key education statistics. It includes statistics on enrollment, graduates, classroom teachers, and expenditures in elementary and secondary schools and institutions of higher education. This year’s edition features data on enrollment, teachers, graduates, and expenditures for the past 14 years and projections to the year 2008.

When we talk about publications, most of us think immediately of paper copies of books, reports, issue briefs, etc. The growth of the World Wide Web, however, is changing our conception of what the word "publication" means. Because NCES’s goal is to provide information to all those who seek it, and because we are unconcerned with generating revenue from sales of publications, the Web offers an ideal medium for presenting our research. For us, the Web allows us to present our research and data to far more people at a relatively small cost. For our customers, the Web allows our customers immediate access to hundreds of our publications at no extra cost, including the flagship publications mentioned above, and allows them to search across publications and databases for particular topics and data. They can download an entire publication, or search or browse online to find particular pieces of information. If you have not already done so, I invite to explore our Website at http://nces.ed.gov.

III. The Role of NCES in Education Policy Making
The amount and variety of information assembled by NCES is truly an academic’s dream, as it could fuel potentially hundreds of dissertations and research papers. But what drives NCES activity, and what has given rise to this virtual ocean of data, goes beyond purely academic research: the obligation to provide the leaders in our government and in our education system with information they can use to make appropriate policy decisions.

Education statistics have always been a valuable tool in government planning and budgeting. For example, Congress uses them to plan federal education programs and to apportion federal funds among the states. Federal agencies, such as the Departments of Defense, Labor, and Commerce, and the National Science Foundation are concerned with the supply of trained manpower coming out of our schools and colleges and also with the subjects being taught there. And state and local officials are concerned with issues such as the supply of teachers and expected populations of students.

But the appetite for education statistics is growing. Increasingly, education is being seen as the key to a wider range of social and economic issues. During this past election season, several national polls indicated that education was the number one priority in the minds of voters. As a result of this increased attention, proposals for education reform abound, some calling for radical changes in response to supposed crises in our education system. With this heightened importance of education policy comes a need not only for more information, but also for information that is targeted toward specific policy questions. Following are just a few of the current policy discussions where NCES data are used:

  • Are U.S. students acquiring the appropriate skills and knowledge in the core academic subject areas?
  • What effect does early childhood education have on future educational achievement?
  • Are funds for education distributed equitably?
  • What is the relationship of spending on education to educational outcomes and economic productivity?
  • What is the relationship of school governance structures (e.g. schools of choice, charter schools, and ability grouping in classes and programs) to educational outcomes?
  • What constitutes quality in curriculum and instruction and what is their relationship to educational outcomes?
  • Do all types of students have an equal opportunity to learn?
  • Are our schools preparing students to be economically competitive in the global economy?
  • Are high school graduates gaining access and choice in their pursuit of higher education, and are they persisting and completing their programs?

In thinking about NCES’s role in addressing these types of policy questions, I cannot over-emphasize the fact that we do not necessarily provide the answers. This is because NCES does not suggest or advocate policy solutions. We can and do provide information used in formulating opinions, but our publications themselves contain no such opinions. To do so would compromise the integrity of our data and go against our organizational mission.

Thus, our role is one of providing policy-relevant information. We do this in several ways. In some cases, we can pull together existing data from several sources and present it in a focused fashion. We do this often in the form of short issue briefs. For our longer term data agenda, however, decisions regarding the topics to address and the methods to address them become more complex. In some cases, we are directed by Congress to conduct research and present data on specific topics and in a specific fashion. Such is the case with the National Assessment of Educational Progress and our relationship with the National Assessment Governing Board (NAGB) that oversees the NAEP program. The majority of our national data, however, is developed within NCES, taking into consideration what we should do and what we can do.

Regarding what we should do, one important factor is of course the potential benefit to our customers. Our data agenda should be relevant to current high-priority policy discussions and should provide data and information that either has not yet been collected or that updates existing knowledge. We gauge the potential benefit to customers based on input from customer satisfaction surveys, the national advisory panels, our own staff expertise, and other, more informal techniques. Another important factor is whether the data agenda is appropriate with NCES’s legislated responsibilities. As a statistical agency, our focus is on collecting and reporting data using sound scientific methods and doing so in an objective manner. If the potential project is not data-based, or cannot be addressed in an objective and methodologically sound manner, then it is not particularly appropriate for our agency. Finally, we must consider whether the research can be more efficiently done by another organization.

Regarding what we can do, we are of course limited by the costs of the project. Costs need to be examined not only in terms of financial resources, but staff resources as well. We also look at the cost of the burden we place on those who would be the subjects of our data collection activities, such as the teacher or administrator responding to the questionnaire or the student participating in the assessment. We are also limited by the feasibility of the project, both in terms of time necessary to complete it, and our capacity to collect and process the data and disseminate the results.

Given these criteria for our national data agenda, I would like to provide just a few examples of how NCES has had a role in policy discussions and how NCES has altered its research agenda to better address policy issues.

Curriculum Design and Instructional Practice

Although the most widely publicized data from the Third International Mathematics and Science Study were perhaps the achievement data, NCES, in conjunction with the National Science Foundation, actively sought to explore two other factors related to achievement: curriculum and instruction. In the curriculum study, researchers examined mathematics and science textbooks and curriculum frameworks of the countries participating in TIMSS. One of the key findings was that textbooks and curriculum frameworks in the United States tend to have more topics than those in other countries. This conclusion has led to examinations at the state, district, and school levels into whether our emphasis on breadth of coverage comprises in-depth understanding of key topics. In the videotape study of age 13 or eighth-grade mathematics instruction in Germany, Japan, and the United States, researchers found that the predominant teaching strategy in Japanese classrooms is to involve students in developing a concept, whereas U.S. teachers were far more likely to simply state the concept. Again, this has led to many discussions at all levels regarding quality educational practices. In neither of these cases did NCES suggest a solution, but in both of these cases, our data has advanced discussions of educational policy.

Assessments of Students with Special Disabilities

NCES is working to improve assessments for special populations who historically have been difficult to assess and sometimes have been entirely excluded from NCES surveys, such as students with disabilities and students with limited English proficiency. We have established that as many as 5% of all students are excluded from national assessments based on a disabling condition and as many as 2% are excluded based on their limited English proficiency.

NCES has addressed these issues recently by:

  • Changing the criteria that local schools use for including students in our surveys and experimenting with ways to encourage school officials to include more students; and
  • Experimenting with providing accommodations for some students such as providing translations of test items and extending time for completing the assessments.

School Violence

Unfortunately, over the past few years in the United States, there have been several tragic incidents of violence at schools committed by students. These events received a great deal of attention in the press and caused many to conclude that school violence is on the rise. Naturally, there were debates regarding appropriate policy responses. To help inform these debates, NCES, in collaboration with the U.S. Department of Justice, developed an Annual Report on School Safety. The first report, issued just this past October, provides a comprehensive view of existing data and showed that overall, school crime is declining, serious violent crime is neither increasing nor decreasing, and that the number of students carrying weapons to school is decreasing. For example, 100% of the serious crime in American public schools is isolated in 10% of the public schools; 90% had no incidence of serious crime in 1997. At the same time, it found evidence of a growing gang presence in schools and noted that more students today are fearful in schools than in the past. Because these data provide a detailed picture of the extent and nature of the problem, they allow policymakers to formulate targeted responses to specific aspects of the problem, such as the recently announced federal initiative aimed at curbing school violence.

Development of Longitudinal Survey Designs to Measure School Effectiveness

For NCES, the challenge of measuring school effectiveness has resulted in efforts to improve the design of longitudinal surveys. Traditionally, studies which follow students over time have resulted in inadequate numbers of students within classrooms or schools for the purposes of estimating school effects, while at the same time providing nationally representative data.

To minimize the limitations and still provide measures of both student and school change, NCES incorporated a number of innovations in the design of the National Educational Longitudinal Study (NELS) including:

  • Adding additional students in successive waves so that the sample remains representative for each wave;
  • Surveying base-year ineligible students and bringing base-year excluded students back into the sample;
  • Sampling during follow-ups to uncover differences from the target sample and potential sample bias;
  • Developing new ways to represent unique populations which add to the data’s accuracy in asking specific questions for population subgroups; and
  • Augmenting students and teachers within the 30 largest statistical metropolitan areas to researchers a more analytically powerful database.

Improvements in Our Data Agenda

Based on growing interest in instructional practices and various reform proposals, our School and Staffing Survey is being redesigned significantly to focus specifically on issues of instructional practice and school reform. It will also include more detailed resource data to enable policymakers to analyze the relationships between fiscal attributes and school, teacher, and student characteristics. A special supplemental component will also study the universe of charter schools in the U.S. Our new data agenda also calls for two major new longitudinal data collections to address the growing interest in early childhood education. The "Birth Cohort" study will include 1000 children born each month in the year 2000 and follow them from birth through age 6. The "Kindergarten Cohort" study, which begins this fall, is focusing on some 20,00 students in 1000 public and private schools across the country as they enter kindergarten and follow them through grade five.

Summary

Our role in informing policy discussions is at the heart of our mission. To fulfill it, we must be responsive to the needs of policymakers and our other customers, but also be independent enough to maintain the integrity of our work and chart a research agenda that is feasible and will remain relevant for the long term.

IV. Possibilities for Cooperation between Taiwan and the United States in Education Statistics
As you here in Taiwan pursue a similar mission and conduct similar research, I believe our two countries have much to gain from increased collaboration in the area of educational statistics.

I am committed to learning from each other’s experiences through participation in cooperative international data strategies. Many people dismiss the usefulness of international education comparisons because of differences between systems in terms of structure and practices. While these differences may make achievement data difficult to compare, many other types of research are made possible because of these differences. For example, the data from TIMSS on Japanese curriculum and instructional practice has been of immense value to educators in the United States. By providing information on alternate ways of doing things, these data have sparked deep reflection on and reexamination of current practice throughout the U.S. and will no doubt result in policy revisions in some districts and States. From other international data, we can observe patterns of achievement in countries where secondary school students pursue specialized programs of study, a practice which is relatively less common in the United States. These data have been useful to policymakers as they have considered various proposals for secondary school reform.

I am certain that data from Taiwan, coupled with more information on educational practices, can be useful to researchers in the United States. In fact, there is already an interest in Taiwanese education, based in part on the research of Harold Stevenson and James Stiegler. Similarly—if I may humbly suggest—I believe that data on the United States education system, with its wide variety of programs and practices, can be of use to researchers and policymakers in Taiwan.

I also believe we both can benefit from sharing research and methodological advancements. Despite the differences between our educational systems, cultures, and political and bureaucratic environments, I believe our experiences in conducting educational research and large-scale statistical studies can be highly instructive for one another. We can share technical information, such as the methodology used to measure a particular aspect of the education system. A set of indicators or a format for reporting data used by one of us might serve as models for the other. Or, one of us might benefit from the other’s experience in trying to present complex data and analyses to audiences interested in clear and uncomplicated answers.

Thus, I hope that in the future there will be increased exchanges between our two countries, including regularly scheduled visits, staff exchanges and increased participation in international studies and conferences.

V. Potential Benefits to Taiwan’s Education System from Participation in International Education Research Projects
Just as I believe Taiwan has much to gain from working with the United States, I believe it has even more to gain from participation in multinational research projects. The collaborative process involved in these studies provides an opportunity not only to learn from the experiences of a much broader range of researchers, institutions, and education systems, but also to be a part of cutting edge research. It would of course be possible to learn from this process even without participating, by simply reading the reports and technical manuals, but the lessons would be far more limited and far less enriching.

One major benefit of participation is that these international assessments can sometimes fill important gaps in individual countries’ own assessment and research strategies. Participation in ongoing international assessments such as TIMSS provides a reliable source of high-quality data that can be compared over time and used to establish valuable international benchmarks. This is extremely useful for those countries that may not conduct their own assessments of particular subjects, at particular grade levels, or using particular types of questions. Similarly, the background information included in the assessments may create a new source of useful data for participating countries. Even with our numerous research activities within the United States, one of our best sources for data on student achievement and background characteristics continues to be our data from these international assessments.

The benefits only increase as innovations in research design evolve with each study. For example, the research into school processes typified by the TIMSS curriculum and videotape studies will become more common in future international educational research. It will be a significant part of TIMSS-R, as well as future OECD efforts. Again, while the results will be valuable to education systems that only observe the results, the benefits will be far greater to those systems that participate, simply because the results will be far more relevant.

Just as international comparative research is beginning to look beyond achievement, it is also moving toward a more sophisticated definition of achievement. One of the goals of the PISA project of the OECD is to develop mechanisms for measuring achievement in broad, cross-curricular areas. These indicators will be valuable to anyone who believes success in life depends on more than subject-specific academic knowledge. Those countries participating will be able to assess their own students on these innovative measures, resulting in information that would not be available otherwise, which will provide valuable supplements to ongoing national assessment programs in the individual countries.

VI. Conclusion
As I mentioned at the beginning of my speech, international education research has progressed far beyond the "horse race" climate that characterized previous projects. The international research community is moving ahead with explorations of achievement in a wider variety of subjects and using a wider variety of methodologies. It is also moving toward a broader definition of achievement and is becoming more focused on influences on achievement. Each country participating stands to benefit from not only assessing its students on these cutting edge measures, but also from learning of the experiences of others in tackling many of the same challenges it may be facing in its own research activities.

I have shared with you today an overview of my organization’s research activities and data agenda. It is a data agenda that has in part been influenced by what we have learned and are continuing to learn from other countries through our participation in international research activities. It is my hope that the future will bring more sharing between our two countries; specifically, sharing of data, ideas, and experiences. In this way, not only will we as researchers benefit, but so will the students of both our nations.

Link To:
NCES Survey and Program Areas