Current Population Survey (CPS) – October Supplement

5. Data Quality and Comparability

SAMPLING ERROR

Although the estimation methods used in the CPS do not produce unbiased estimates, biases for most estimates are believed to be small enough so that the confidence interval statements are approximately true. Standard error estimates are computed using replicate variance techniques and reflect contributions not only from sampling error but also from some types of nonsampling error, particularly response variability and intra-interviewer correlation. Because replicate variance techniques are somewhat cumbersome, simplified formulas called generalized variance functions (GVFs) have been developed for various types of labor force characteristics. The GVF can be used to approximate an estimate’s standard error, but this only indicates the general magnitude of its standard error rather than a precise value. Standard error estimates computed using generalized variance functions are provided in Employment and Earnings and other BLS publications. 

Nonsampling Error

Although the full extent of nonsampling error in the CPS is unknown, special studies have been conducted to quantify some of the possible sources. The effect of nonsampling error should be small on estimates of relative change, such as month-to-month change. Estimates of monthly levels would be more severely affected by nonsampling error.

Coverage Error. The concept of coverage in the survey sampling process is the extent to which the total population that could be selected for the sample “covers” the survey’s target population. Undercoverage in the CPS results from missed housing units and missed persons within sample households. Overall CPS undercoverage for households was estimated to be about 10 percent for October 2005 and about 11 percent for October 2006. It is known that the CPS undercoverage varies with age, sex, race, and Hispanic origin. Generally, undercoverage is larger for men than for women and larger for Blacks, Hispanics, and other races than for Whites. Ratio adjustment to independent age/sex/race/origin population controls, as described previously, partially corrects for the biases due to survey undercoverage. However, biases exist in the estimates to the extent that missed persons in missed households or missed persons in interviewed households have different characteristics than interviewed persons in the same age/sex/race/origin group.

The independent population estimates used in the estimation procedure may be a source of error although, on balance, their use substantially improves the statistical reliability of many of the figures. Errors may arise in the independent population estimates because of underenumeration of certain population groups or errors in age reporting in the decennial census (which serves as the base for the estimates) or similar problems in the components of population change (mortality, immigration, etc.).

Top

Nonresponse Error.

Unit Nonresponse. Unit nonresponse may have a number of components. A respondent may refuse to participate in the survey, may not be capable of completing the interview, or may not be available to the interviewer during the specified survey period. If the entire household does not participate, this situation is referred to as a “Type A noninterview.” There is also another type of (partial) unit nonresponse, namely, that one or more individual persons within the household refuse to be interviewed. This is not a major problem in the CPS since any responsible adult may be able to report information for other persons as a proxy reporter. There are other variations on unit nonresponse; detailed consideration of these may be found in The Current Population Survey: Design and Methodology (Technical Paper 66) (U.S. Department of Commerce 2006).

For the October 2005 basic CPS, the nonresponse rate was 7.4 percent, and the nonresponse rate for the October supplement was an additional 3.4 percent. These two nonresponse rates led to a combined nonresponse rate of 10.5 percent. For the October 2006 basic CPS, the household-level nonresponse rate was 8.1 percent, and the person-level nonresponse rate for the October supplement was an additional 3.9 percent. Since the basic CPS nonresponse rate was a household-level rate and the School Enrollment supplement nonresponse rate was a person-level rate, these rates couldn’t be combined to derive an overall nonresponse rate. Since it is unlikely the nonresponding households to the basic CPS had the same number of persons as the households successfully interviewed, combining these rates would have resulted in an overestimate of the “true” person-level overall nonresponse rate for the October supplement (for more information, see The Current Population Survey October 2006: School Enrollment Supplement Technical Documentation, U.S. Department of Commerce 2006).

Item Nonresponse. Although an imputation procedure is implemented for item nonresponse in the CPS, there is no way of ensuring that the errors of item imputation will balance out and that any potential bias has been avoided.

Measurement Error. The main sources of nonsampling variability in the responses to the October Supplement are those inherent in the survey instrument. The question of current school enrollment may not be answered accurately for various reasons. Some respondents may not know current grade information for every student in the household, a problem especially prevalent for households with members in college or in nursery school. Confusion over college credits or hours taken by a student may make it difficult to determine the year in which the student is enrolled. Problems may occur with the definition of nursery school (a group or class organized to provide educational experiences for children), where respondents’ interpretations of “educational experiences” vary. 

Top

Data Comparability

NCES collects preschool, elementary school, secondary school, and postsecondary education enrollment and completion data through a wide range of studies including the National Household Education Surveys Program (NHES, see NHES chapter), the Common Core of Data (CCD, see CCD chapter), the Private School Survey (PSS, see PSS chapter), the Integrated Postsecondary Education Data System (IPEDS, see IPEDS chapter), and the National Postsecondary Student Aid Study (NPSAS, see NPSAS chapter). In addition, the Bureau of the Census collects the American Community Survey (ACS), which is another household survey that includes some school enrollment and educational attainment data.

Because of differences in data collection modes, respondent selection, interviewer training, collection and reference periods, and differing survey processes, data obtained from the CPS and other sources are not entirely comparable. This is an example of nonsampling variability that is not reflected in the standard errors. Therefore, caution should be used when comparing results from different sources.

Top