Technical Report:

Adjusting for Coverage Bias Using Telephone Service Interruption Data

December 1996

Highlights

The National Household Education Survey (NHES) is a data collection system of the (NCES), which has as its legislative mission the collection and publication of data on the condition of education in the Nation. The NHES is specifically designed to support this mission by providing information on those educational issues that are best addressed by contacting households rather than schools or other educational institutions. The NHES provides descriptive data on the educational activities of the U.S. population and offers policymakers, researchers, and educators a variety of statistics on tbe condition of education in the United States.

The NHES is a telephone survey of the noninstitutionalized civilian population of the U.S. Households are selected for the survey using random digit dialing (RDD) methods, and data are collected using computer-assisted telephone interviewing (CATI) procedures. These procedures provide a cost effective means for quickly surveying households with telephones. Approximately 60,000 households are screened for each administration, and individuals within households who meet predetermined criteria are sampled for more detailed or extended interviews. The data are weighted to permit estimates of the entire population. Tbe NHES survey for a given year typically consists of a Screener, which collects household composition and demographic data, and extended interviews on two substantive components addressing education-related topics. In order to assess data item reliability and inform future NHES surveys, each administration also includes a subsample of respondents for a reinterview.

The primary purpose of the NHES is to conduct repeated measurements of the same phenomena at different points in time, although one-time surveys on topics of interest to the Department of Education are also conducted. This has been done by repeating topical components on a rotating basis to provide comparative data across survey years. In addition, each administration of the NHES has benefited from experiences with previous cycles, resulting in enhancements to the survey procedures and content. Thus, while the survey affords the opportunity for tracking phenomena across time, it is also dynamic in addressing new issues and including conceptual and methodological refinements.

A new design feature of the NHES prugram implemented in the NHES:96 is the collection of demographic and educational information on members of all screened households, rather than just those households potentially eligible for a topical component. In addition, this expanded screening feature includes a brief set of questions on an issue of interest to education program administrators or policymakers. The total Screener sample size is sufficient to produce state estimates of household characteristics for the NHES:96.

Full-scale implementations of the NHES have heen conducted in 1991, 1993, 1995, and 1996. Topics addressed by the NHES:91 were early childhood education and adult education. The NHES:93 collected information about school readiness and school safety and discipline. The 1991 components were repeated for the NHES:95, addressing early childhood program participation and adult education. Both components underwent substantial redesign to incorporate new issues and develop new measurement approaches. In the NHES:96, the topical components are parent/family involvement in education and civic involvement. The NHES:96 expanded screening feature includes a set of questions on public library use.

In addition to its topical components, the NHES system haa also included a number of methodological investigations. These have resulted in technical reports and working papers covering diverse topics such as telephone undercoverage bias, proxy reporting, and sampling methods. This series of technical reports and working papers provides valuable information on ways of improving the NHES, and may be useful to survey researchers more generally.

This report is a continuation of research on issues related to biases that result from the inability to survey persons who live in households without telephones. Two of the earlier NHES technical reports (Brick, Burke, and West 1992; Brick and West 1992) addressed this important subject. Another bias study involved adding certain questionnaire items to the NHES:93 to evaluate a different method of adjusting the estimates to reduce the bias associated with sampling only persons living in households with telephones. The method involves using data on interruptions of telephone service to adjust the weights of the respondents to the survey. The weights for households that report experiencing some periods of not having telephone service during the twelve months prior to the interview are increased whereas households reporting no breaks in telephone service receive their normal weights. The assumption behind this procedure is that households with interrupted telephone service are more like those without telephones than other telephone households. Although the goal of these adjustments is to reduce the bias due to excluding households without telephones at the time of the survey, a consequence of the adjustments is that the variances of the estimates increase. This analysis examines the benefits of the bias reduction in light of the variance increases and suggests situations in which the adjustments might be beneficial.

The next section provides background information on telephone coverage bias, its implications for estimates from a survey such as the NHES, and previous research using data on telephone service interruptions to reduce coverage bias. Subsequent sections describe the estimates from the NHES:93 of the percentage of persons that experienced some interruption of telephone service, the procedures used to adjust the survey weights using these data, and the statistical implications of using the adjusted weights. The final section summarizes the findings and contains recommendations for use of this technique.

Download/view the full report in a PDF file. (file size: 2,317K)

Help with PDF files

For more information about the content of this report, contact Kathryn Chandler at Kathryn.Chandler@ed.gov.