Skip Navigation

Statistical Standards
Statistical Standards Program
 
Table of Contents
 
Introduction
1. Development of Concepts and Methods
2. Planning and Design of Surveys
3. Collection of Data

 
3-1 Coverage for Frames and Samples
3-2 Achieving Acceptable Response Rates
3-3 Monitoring and Documenting Survey Contracts
3-4 Documenting a Survey System

4. Processing and Editing of Data
5. Analysis of Data / Production of Estimates or Projections
6. Establishment of Review Procedures
7. Dissemination of Data
 
Glossary
Appendix A
Appendix B
Appendix C
Appendix D
 
Publication information

For help viewing PDF files, please click here
COLLECTION OF DATA

SUBJECT: COVERAGE FOR NCES FRAMES AND SAMPLES

NCES STANDARD: 3-1

PURPOSE: To ensure that necessary steps are taken to develop and maintain data collections that are used as sampling frames, and that coverage of sampling frames is evaluated and documented.

KEY TERMS: capture/recapture, confidentiality, coverage, coverage error, dual-frame estimation, estimation, frame, frame population, freshening, half-open interval, multiplicity estimation, noncoverage, overcoverage, supplemental area frame, survey system, target population, undercoverage, and variance.


STANDARD 3-1-1:
Staff responsible for NCES data collections that serve as sampling frames for other NCES surveys must evaluate the coverage of the frame and document coverage rates at least once every 5 years.

    GUIDELINE 3-1-1A: Frames can be retrospectively compared against alternative frames found inside and outside of the Department of Education; considering total list count comparisons, matching operations, and dual-frame estimation procedures using capture/recapture procedures to estimate noncoverage; and providing an estimation of missing units.

    GUIDELINE 3-1-1B: Staff responsible for NCES data collections that are used as sampling frames should maintain two-way communications with survey staff who use their collection as a frame. Procedures such as sharing preliminary data files with survey staff in order to develop frames may be instituted. (For example, staff that use an administrative list of public schools for their frames should be alerted when new data are available and each time there is a major change in the list.)


STANDARD 3-1-2: NCES data collections that are used as sampling frames for other NCES surveys must strive for coverage rates in excess of 95 percent overall and for each major stratum.


STANDARD 3-1-3: Staff using NCES frames for sample surveys must be cognizant of coverage issues, and must take the steps necessary to provide satisfactory coverage for the sample survey. If there is not evidence of a coverage rate of at least 85 percent of the target population, then frame enhancements such as frame supplementation or dual frame estimation must be incorporated into the survey study design.

    GUIDELINE 3-1-3A: The first time a survey is conducted, background design and coverage work should be done before choosing the frame. Alternative frames, if applicable, should be considered and compared.

    GUIDELINE 3-1-3B: Coverage errors such as over- and under-coverage, bad contact information, classification, temporal errors, and other listing errors should be minimized before the use of a frame. Techniques such as list supplements, multiciplicity estimation, half-open intervals, and un-duplication can be used to reduce these errors and improve coverage of the frame.

    GUIDELINE 3-1-3C: Any possible changes to frame variables identified by sample survey staff should be reported to the staff responsible for the data collection being used as the frame. For example, the relevant variables to maintain and consider include: 1) eligibility (e.g., grade span); 2) contact information (e.g., name, address, and phone number); 3) classification variables (e.g., state and school level); and 4) measures of size (e.g., grade enrollment).

    GUIDELINE 3-1-3D: To reduce coverage error, whenever a frame has important deficiencies with respect to the measurement unit, dual-frame estimation should be considered to correct these deficiencies. Since dual-frame estimation can be expensive, the effect dual-frame estimation has on increasing the variance estimates should also be considered when deciding to use dual-frame estimation.


STANDARD 3-1-4: For each sample survey, a description of the frame and its coverage must be included in the survey documentation. This description must include, but is not limited to, the target and frame populations (and exclusions thereof); the name and date of the data collection which provided the original frame; any supplementing done to the original frame; limitations of the frame including the timeliness of the frame; and, if applicable, an estimation of the missing units on the frame.

    GUIDELINE 3-1-4A: Sample survey documentation should include a discussion of coverage issues such as alternative frames that were considered, what was done to improve the coverage of the frame, and how data quality and item non-response on the frame may have affected the coverage of the frame.

    GUIDELINE 3-1-4B: Survey documentation should include any estimation techniques used to improve the coverage of estimates. This would include post-stratification procedures. (For example, a telephone survey could post-stratify estimates of all individuals to account for the exclusion of those without telephones.)

    GUIDELINE 3-1-4C: NCES survey staff should archive their survey's sampling frames as part of the documentation of the survey system found in Standard 3-4, taking security precautions consistent with confidentiality laws into account. This archiving may be particularly important if a preliminary file was used to develop the frame, or if there is a chance that the frame may be used in the future to further develop research questions.


STANDARD 3-1-5: NCES survey staff that use NCES data collections as a frame must share any coverage or usage issues with the NCES data collection staff so that the coverage can be improved for future uses. This standard is related to Guidelines 3-1-3B and 3-1-3C. (For example, after the survey is complete, the survey staff should provide a memo to the NCES data collection staff for the data collection used as a frame, reviewing the major limitations of the coverage or the data quality issues identified.)