NCES Quarterly, Winter 2001: Invited Commentary: Household Data in the Federal Statistical System: The Role of the National Household Education Surveys Program

View Quarterly by: This Issue | Volume and Issue | Topics

Vol 3, Issue 4, Topic: Featured Topic: National Household Education Surveys Program

Invited Commentary: Household Data in the Federal Statistical System: The Role of the National Household Education Surveys Program

By: Susan Schechter, Senior Statistician, Office of Management and Budget

This commentary represents the opinions of the author and does not necessarily reflect the views of the Office of Management and Budget or the National Center for Education Statistics.

The Federal Statistical System
The Value of Household Surveys
Topics and Goals of NHES
Benefits and Drawbacks of Conducting NHES by Telephone
Conclusion
Reference

In order to meet ever-increasing demands to carry out its responsibilities efficiently and effectively, the federal government continues to have a pressing need for data that are timely, accurate, reliable, and relevant. To inform decisions about a vast array of social, economic, housing, and educational services, federal agencies collect, analyze, use, and disseminate a wealth of information.

Much of this information is collected directly from the public—from individuals, large and small businesses, educational and nonprofit institutions, federal contractors, and state, local, and tribal governments. Narrowing the focus of this discussion to education statistics, data typically are collected by asking for information from schools (including individual schools, school districts, and state school systems), teachers, administrators, parents and, of course, students. The National Center for Education Statistics (NCES) has, over the years, made great strides in refining, improving, and expanding its family of surveys in order to create and utilize a far-reaching set of statistics that offer policymakers, researchers, and educators the pertinent information they need. Embracing an approach of continuous examination and evaluation of the methods and procedures used to collect the data can lead to substantial improvements and strengthening of the education statistics we have come to rely upon.

The two reports highlighted in this issue of the Education Statistics Quarterly offer an excellent view of the success, value, and contributions of the NCES-sponsored National Household Education Surveys Program (NHES). In the first report, Efforts by Public K-8 Schools to Involve Parents in Children’s Education: Do School and Parent Reports Agree?, Xianglei Chen offers a valuable comparison of the opinions and views held by both schools and parents. By looking at the level of agreement between schools’ and parents’ reports, we can identify areas that are working well, and also those in need of some improvement. Because parental involvement is considered to be crucial in support of children’s educational development, and because schools devote considerable resources to promoting parental involvement, the results of this study are an important asset to the education community.

In the second report, Participation Trends and Patterns in Adult Education: 1991 to 1999, Sean Creighton and Lisa Hudson provide important information on the degree to which adults are actively engaged in educational activities and examine trends over time to view changing patterns of participation. This study takes a careful look at six types of adult education activities and offers detailed analyses of the characteristics of participants. The key finding of this study—that participation rates in adult education increased for virtually every group of adults examined—is truly a “good news” story that reveals tremendous accomplishment in providing greater access to educational opportunities. This very positive result, however, is tempered by a detailed view of those groups that traditionally have had relatively low rates of participation in adult education. For example, the study found that Hispanics, those with lower levels of education, those with lower status jobs, and those employed part time all continued to have relatively low rates of participation in work-related adult education at the end of the 1990s. Both the positive and negative findings of this study are critically important as adult education planners develop new strategies for the coming years.

These two studies have considerable merit in and of themselves and could easily serve as the subject of extensive commentary on their own. However, it is also useful to discuss more broadly the federal statistical system and the role that NHES and other national household surveys play in providing our country with information needed for policy formulation, program evaluation and assessment, and decisionmaking. Part of this discussion involves the telephone survey methodology that is used in NHES as well as other national surveys.

The Federal Statistical System

The United States is one of a small number of countries that have highly decentralized national statistical systems. More than 70 federal agencies, or organizational units within agencies, collect statistical information, often in concert with program administration or regulatory functions. The Office of Management and Budget (OMB) provides oversight, coordination, and guidance for federal statistical activities and promotes the quality, integrity, and efficiency of federal government statistical programs. In particular, OMB works closely with federal agencies to improve the relevance, accuracy, timeliness, and availability of federal statistics while protecting the integrity of statistical information products, respecting pledges of confidentiality, and minimizing both the reporting burden on the public and the statistical system’s use of federal resources.

To ensure the quality of federal government statistical activities, careful attention is paid to the underlying statistical methods and procedures that accompany any information collection. Strengthening source data to improve their coverage, accuracy, timeliness, and quality is a goal shared by the federal statistical community at large. While considerable progress has been made in improving the overall performance and efficiency of the federal statistical system as well as the quality of the data provided by specific studies, rapid changes in our economy and society present continuing challenges to our statistical infrastructure and the methods used to obtain needed data.

The Value of Household Surveys

The surveys and censuses that support the infrastructure of the federal statistical system incorporate a wide variety of methods, procedures, and analytic approaches. The data collection methods for a specific survey are usually tailored to meet data needs and study objectives within resource and time constraints. While some data collections measure particular phenomena or are only one-time surveys, many other federal surveys are ongoing, national in scope, and serve to describe and measure important social, economic, housing, and educational dimensions of the United States.

NHES joins other household-based federal surveys—the Current Population Survey, the Consumer Expenditures Survey, the Survey of Income and Program Participation, the National Health Interview Survey, the National Immunization Survey, the National Crime Victimization Survey, and the American Housing Survey, to name just a few—in providing key indicators on critical aspects of our society. These surveys all share one important feature: they collect information from a representative sample of the U.S. population through the administration of questionnaires to household members. The voluntary participation by literally millions of people in such federal surveys directly supports the federal statistical system and is critical to the ultimate quality of the information that federal agencies produce.

Topics and Goals of NHES

The majority of national education statistics come from institution-based surveys. NHES provides the only regularly fielded education surveys that target household members. As a system of household surveys, NHES has the capability to identify, describe, and address a wide range of education-related issues that are not easily covered by surveys of institutions. For example, NHES provides information about activities that families engage in with young children that might promote these children’s readiness to begin school. Most topics covered by NHES are repeated in various survey years on a rotating basis.

Included in the ongoing NHES data collection system are surveys on school readiness, early childhood program participation, parent and family involvement in education, before- and after-school programs and activities, and adult education. A particularly attractive feature of NHES, shared by many other major national surveys, is that by conducting the surveys on a repeated basis over the course of years, it provides measures of the same phenomena at different points in time. These trend analyses are very important, as they detect significant change in patterns and practice. However, NHES also has the flexibility to include one-time surveys on key topics when the need arises; for example, the 1993 collection included a survey on household members’ perceptions of school safety and discipline.

NHES is designed to produce reliable estimates not only for the total U.S. noninstitutionalized population, but also for different racial/ethnic groups. Estimates by race and ethnicity are of great interest, especially for monitoring trends over time. Therefore, the NHES sample design oversamples minorities in order to increase the reliability of estimates for these groups.

Each collection of NHES begins with the screening of a representative sample of households to select participants for that year’s topical surveys. The number of households screened has ranged from 45,000 to 64,000. The high costs associated with screening large numbers of households in order to meet the sample size requirements of NHES have led to a design that allows for more than one topical survey to be carried out concurrently whenever possible. In deciding which topics should be addressed in the same collection, consideration is given to the probability of households being eligible for one or more of the topical surveys. The ideal combination of topical surveys is one that maximizes the probability of a household qualifying for a survey interview, but limits the number of households that must respond to more than one survey.

Benefits and Drawbacks of Conducting NHES by Telephone

The design of data collection methods depends on numerous factors, including the objectives of the study and the type of information sought, the length and complexity of the questionnaire, the resources available, and the urgency with which the data are needed. Any choice of data collection mode and its accompanying procedures must weigh heavily on the quality and efficiency of a project. Because of their complexity and length, most national surveys sponsored by the federal government are conducted by personal visits.

The choice to conduct a survey by telephone is typically made because the results can be quickly produced, the cost is less than a personal visit, and the survey instrument is adaptable to telephone administration (i.e., the length is not terribly great, and hand cards, calendars, and other administration tools are not needed to improve response quality). For these reasons, the telephone was chosen as the mode of administration for NHES. Households are selected for screening using list-assisted random-digit-dialing (RDD) methods, and data are collected using computer-assisted telephone interviewing (CATI) procedures.

While using the telephone as the mode of survey administration has numerous benefits, it is generally held that, compared to personal-visit surveys, certain types of survey errors will be higher for telephone surveys. Two types of errors that tend to be higher for telephone surveys are nonresponse bias and coverage bias. Nonresponse bias occurs when a significant number of the people sampled do not respond to the questionnaires and are different from those who do in a way that is important to the study; coverage bias occurs when the list or frame from which the sample is drawn does not include all elements of the population that the researchers wish to study (Salant and Dillman 1994).

When taking the household screening interview as well as the completed topical interviews into account, NHES has an overall response rate below 70 percent. With the advent of answering machines, cell phones, caller ID, and other technologies, it is unclear at this point how response rates for telephone surveys such as NHES will be affected and whether concerns about survey error will grow. However, NHES does have a comprehensive and sophisticated approach to addressing possible biases that might result from coverage limitations or nonresponse.

Conclusion

The family of surveys conducted by NCES is designed to address the needs of the education community and to provide accurate, timely, reliable, high-quality data for education policymakers, practitioners, and the general public. NHES is a critical component of this family of surveys, as it provides household data on a wide array of important topics. It is an excellent example of a well-designed survey that takes aggressive action to minimize nonresponse and coverage biases (as well as other types of survey errors). Household surveys such as NHES constitute a key component of the federal statistical system, as they provide a portrait of our nation’s social, economic, housing, and educational characteristics. Ongoing research and evaluation efforts to improve the quality of all federal data, including those provided by NHES, will continue as our country’s demand for relevant information increases. These efforts are laudable and will serve the nation, as well as the federal statistical system, well.

Reference

Salant, P., and Dillman, D. (1994). How to Conduct Your Own Survey. New York: Wiley and Sons.