- Introduction
- Adult Literacy and Lifeskills Survey (ALL)
- Academic Libraries Survey (ALS)
- Baccalaureate and Beyond (B&B) Longitudinal Study
- Beginning Postsecondary Students (BPS) Longitudinal Study
- Beginning Teacher Longitudinal Study(BTLS)
- Common Core of Data (CCD)
- Current Population Survey (CPS)
- Overview
- Uses of Data
- Key Concepts
- Survey Design
- Data Quality and Comparability
- Contact Information
- Methodology and Evaluation Reports
- CPS Website
- Download PDF
- Early Childhood Longitudinal Study, Birth Cohort (ECLS-B)
- Early Childhood Longitudinal Study, Kindergarten Class of 1998‑99 (ECLS‑K)
- Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS‑K:2011)
- Educational Longitudinal Study of 2002 (ELS 2002)
- Quick Response Information System (FRSS/PEQIS)
- High School and Beyond (HS&B) Longitudinal Study
- High School Longitudinal Study of 2009 (HSLS:09)
- High School Transcript Studies (HSTS)
- International Adult Literacy Survey (IALS)
- Integrated Postsecondary Education Data System (IPEDS)
- Middle Grades Longitudinal Study of 2017– 18 (MGLS:2017)
- National Assessment of Adulty Literacy (NAAL)
- National Assessment of Educational Progress (NAEP)
- National Adult Literacy Survey (NALS)
- National Education Longitudinal Study of 1988 (NELS 88)
- National Household Education Survey Program (NHES)
- National Longitudinal Study of the High School Class of 1982 (NLS 72)
- National Postsecondary Student Aid Study (NPSAS)
- National Study of Postsecondary Faculty (NSOPF)
- National Teacher and Principal Survey (NTPS)
- Program for the International Assessment of Adult Competencies (PIAAC)
- Progress In International Reading Literacy Study (PIRLS)
- Program For International Student Assessment (PISA)
- Private School Universe Survey (PSS)
- School and Staffing Survey (SASS)
- SASS Principal Follow-Up Survey (PFS)
- SASS School Library Media Center Survey (SLS)
- SASS Teacher Follow-up Survey (TFS)
- Survey of Earned Doctorates (SED)
- Crime and Safety Surveys (SCS & SSOCS)
- Teaching and Learning International Survey (TALIS)
- Trends In International Mathematics and Science Study (TIMSS)
Current Population Survey (CPS) – October Supplement
5. Data Quality and Comparability
SAMPLING ERROR
Although the estimation methods used in the CPS do not produce unbiased estimates, biases for most estimates are believed to be small enough so that the confidence interval statements are approximately true. Standard error estimates are computed using replicate variance techniques and reflect contributions not only from sampling error but also from some types of nonsampling error, particularly response variability and intra-interviewer correlation. Because replicate variance techniques are somewhat cumbersome, simplified formulas called generalized variance functions (GVFs) have been developed for various types of labor force characteristics. The GVF can be used to approximate an estimate’s standard error, but this only indicates the general magnitude of its standard error rather than a precise value. Standard error estimates computed using generalized variance functions are provided in Employment and Earnings and other BLS publications.
Nonsampling Error
Although the full extent of nonsampling error in the CPS is unknown, special studies have been conducted to quantify some of the possible sources. The effect of nonsampling error should be small on estimates of relative change, such as month-to-month change. Estimates of monthly levels would be more severely affected by nonsampling error.
Coverage Error. The concept of coverage in the survey sampling process is the extent to which the total population that could be selected for the sample “covers” the survey’s target population. Undercoverage in the CPS results from missed housing units and missed persons within sample households. Overall CPS undercoverage for households was estimated to be about 10 percent for October 2005 and about 11 percent for October 2006. It is known that the CPS undercoverage varies with age, sex, race, and Hispanic origin. Generally, undercoverage is larger for men than for women and larger for Blacks, Hispanics, and other races than for Whites. Ratio adjustment to independent age/sex/race/origin population controls, as described previously, partially corrects for the biases due to survey undercoverage. However, biases exist in the estimates to the extent that missed persons in missed households or missed persons in interviewed households have different characteristics than interviewed persons in the same age/sex/race/origin group.
The independent population estimates used in the estimation procedure may be a source of error although, on balance, their use substantially improves the statistical reliability of many of the figures. Errors may arise in the independent population estimates because of underenumeration of certain population groups or errors in age reporting in the decennial census (which serves as the base for the estimates) or similar problems in the components of population change (mortality, immigration, etc.).
Nonresponse Error.
Unit Nonresponse. Unit nonresponse may have a number of components. A respondent
may refuse to participate in the survey, may not be capable of completing the interview,
or may not be
available to the interviewer during the specified survey period. If
the entire household does not participate, this situation is referred to as a “Type
A noninterview.” There is also another type of (partial) unit nonresponse, namely,
that one or more individual persons within the household refuse to be interviewed.
This is not a major problem in the CPS since any responsible adult may be able to
report information for other persons as a proxy reporter. There are other variations
on unit nonresponse; detailed consideration of these may be found in
The Current Population Survey:
Design and
Methodology (Technical Paper
66) (U.S. Department of Commerce 2006).
For the October 2005 basic CPS, the nonresponse rate was 7.4 percent, and the nonresponse rate for the October supplement was an additional 3.4 percent. These two nonresponse rates led to a combined nonresponse rate of 10.5 percent. For the October 2006 basic CPS, the household-level nonresponse rate was 8.1 percent, and the person-level nonresponse rate for the October supplement was an additional 3.9 percent. Since the basic CPS nonresponse rate was a household-level rate and the School Enrollment supplement nonresponse rate was a person-level rate, these rates couldn’t be combined to derive an overall nonresponse rate. Since it is unlikely the nonresponding households to the basic CPS had the same number of persons as the households successfully interviewed, combining these rates would have resulted in an overestimate of the “true” person-level overall nonresponse rate for the October supplement (for more information, see The Current Population Survey October 2006: School Enrollment Supplement Technical Documentation, U.S. Department of Commerce 2006).
Item Nonresponse. Although an imputation procedure is implemented for item nonresponse in the CPS, there is no way of ensuring that the errors of item imputation will balance out and that any potential bias has been avoided.
Measurement Error. The main sources of nonsampling variability in the responses to the October Supplement are those inherent in the survey instrument. The question of current school enrollment may not be answered accurately for various reasons. Some respondents may not know current grade information for every student in the household, a problem especially prevalent for households with members in college or in nursery school. Confusion over college credits or hours taken by a student may make it difficult to determine the year in which the student is enrolled. Problems may occur with the definition of nursery school (a group or class organized to provide educational experiences for children), where respondents’ interpretations of “educational experiences” vary.
Data Comparability
NCES collects preschool, elementary school, secondary school, and postsecondary education enrollment and completion data through a wide range of studies including the National Household Education Surveys Program (NHES, see NHES chapter), the Common Core of Data (CCD, see CCD chapter), the Private School Survey (PSS, see PSS chapter), the Integrated Postsecondary Education Data System (IPEDS, see IPEDS chapter), and the National Postsecondary Student Aid Study (NPSAS, see NPSAS chapter). In addition, the Bureau of the Census collects the American Community Survey (ACS), which is another household survey that includes some school enrollment and educational attainment data.
Because of differences in data collection modes, respondent selection, interviewer training, collection and reference periods, and differing survey processes, data obtained from the CPS and other sources are not entirely comparable. This is an example of nonsampling variability that is not reflected in the standard errors. Therefore, caution should be used when comparing results from different sources.