Technical Report:

Reinterview Results for the School Safety and Discipline and School Readiness Components

June 1997

Highlights

This report examines the reliability of the responses to the interviews in both the School Safety & Discipline (SS&D) and School Readiness (SR) components of the NHES:93. Estimates from these components, like the estimates from every sample survey, are subject to both sampling error and nonsampling error. Sampling errors, the differences between the population values and the sample estimates that arise because data are obtained from only a sample of the population, are generally well understood and can be estimated from the survey data themselves. Nonsampling errors, on the other hand, arise from a variety of sources and are more difficult to measure. Important components of nonsampling error for the NHES:93 include coverage, nonresponse, and measurement errors.

In this analysis, measurement errors are estimated by reinterviewing a sample of respondents and asking them a subset of the same items included in the original interview. The reinterview procedure does not account for all the measurement errors in the interviewing process. For example, systematic errors that would be made in both the original interview and the reinterview are not discovered with this approach. Examples of systematic errors are respondents consistently underreporting income by excluding interest income, or respondents reporting that no students brought weapons to school, a possible social desirability bias. In contrast, the statistics produced by comparing the original and reinterview responses estimate the consistency of reporting, assuming both interviews were conducted under the same general conditions.

A reinterview was conducted for the early childhood component of the NHES:91 (Brick and West 1992) The approach to the reinterview for the NHES:93 is similar to that study, but the methods used expand on those in the NHES:91. The general review of the design and analysis of reinterviews presented by Forsman and Schreiner (1991) is useful background for understanding the goals and methods used in the NHES:93 reinterview. Brick et al. (1994a) discuss the use of reinterview data in the broader context of other nonsampling errors.

In the NHES:93 reinterview study, the same respondents were asked to respond to the same items on different occasions. They may have given different responses at these two times. Such discrepancies in responses can be grouped into four categories:

Circumstances may have changed between the first and second interview and both answers, although different, may be correct;

The original data item may have been recorded incorrectly (interviewer error) or reported incorrectly (respondent error).

The reinterview data item may have been recorded or reported incorrectly.

Both the original and reinterview responses may have been recorded or reported incorrectly.

In the NHES:93 reinterviews, discrepancies between some of the original interview items and the reinterview items were reconciled. This means that when the reinterview response was different from the original response, the interviewer asked the respondent to verify which response was correct. This process of reconciling the responses was done after the reinterview was completed, and it was done only for selected items. For items that were reconciled, there are three different responses available for analysis: the original response, the unreconciled reinterview response, and the reconciled response when the original and reinterview responses were not identical.

In this report, all three of these responses are used. Typically, the difference between the original response and unreconciled reinterview response is used to assess the consistency of reporting, while the difference between the original and reconciled responses is used to assess the bias in the estimate from the original survey. The suitability of the reconciled data for estimating response bias is dependent upon the reconciled response being more accurate than the original and reinterview response (Forsman and Schreiner 1991). This condition is examined using the NHES:93 data later in this report.

An objective in many reinterview programs is to provide a check on interviewers who might be recording entire interviews without speaking to the respondents (U.S. Department of Commerce 1968). Since the NHES:93 was a CATI survey operated in a centralized location, there was no need to design reinterviews to verify that interviews were genuine. The CATI interviews were closely monitored, and it was highly unlikely that a telephone interviewer could invent whole interviews.

The primary objectives for the NHES:93 reinterview program were:

To identify survey items that were not reliabe, i.e., the two interviews did not elicit the same response;

To quantify the magnitude of the response variance for groups of items collected from the same respondent at two different times; and

To provide feedback to improve the design of questionnaire items for future surveys.

A subset of the original SS&D and SR questionnaire items was included in the reinterview program for the NHES:93 to reduce the burden on respondents who had already completed one or more full interviews and to prevent asking some questions that were dependent on the specific date of the interview. In general, the items selected were those that were very important substantively, were not highly time dependent, and were not already examined in previous NHES reinterviews. The reinterview questionnaires are reproduced in full in Appendix A.

Items were selected from specific subject areas. Those subject areas chosen for the reinterview and the associated populations for each are as follows:

SS&D Reinterview Subject Areas:

School characteristics (all parents)
School environment (all)
School safety (all)
School discipline policy (all)
Tobacco, alcohol, and other drugs (all)
Alcohol and drug education (all)
Child characteristics (all parents)
Family characteristics (all parents)
Community characteristics (all)

SR Reinterview Subject Areas:

Developmental profile (preschoolers only)
Early childhood programs (all)
Child adjustment to kindergarten or primary school (kindergarten and primary only)
Teacher feedback on child's performance and behavior (kindergarten and primary only)
Special help in school (primary only)
Health and nutrition (all)
Entry to kindergarten (kindergarten and primary)
Reading and television (all)

Some of the key features of the study design of the NHES:93 reinterview program are described below. The properties of the statistics computed using the reinterview responses are then described and these statistics are presented for the SS&D and SR components of the NHES:93. The final section of the report summarizes the findings and provides some recommendations for future work.

Download/view the full report in a PDF file.(399K)

Help with PDF files

For more information about the content of this report, contact Kathryn Chandler at Kathryn.Chandler@ed.gov.