Skip Navigation

Statistical Standards
Statistical Standards Program
 
Table of Contents
 
Introduction
1. Development of Concepts and Methods
2. Planning and Design of Surveys
3. Collection of Data

 
3-1 Coverage for Frames and Samples
3-2 Achieving Acceptable Response Rates
3-3 Monitoring and Documenting Survey Contracts
3-4 Documenting a Survey System

4. Processing and Editing of Data
5. Analysis of Data / Production of Estimates or Projections
6. Establishment of Review Procedures
7. Dissemination of Data
 
Glossary
Appendix A
Appendix B
Appendix C
Appendix D
 
Publication information

For help viewing PDF files, please click here
COLLECTION OF DATA

SUBJECT: MONITORING AND DOCUMENTING SURVEY CONTRACTS

NCES STANDARD: 3-3

PURPOSE: To assist NCES staff in monitoring and documenting survey contract activities.

KEY TERMS: edit, estimation, imputation, response rate, stage of data collection, survey system, and variance.


STANDARD 3-3-1: The Contracting Officer's Representative (COR) must work to ensure that the contractor meets: (a) contract specifications, (b) contract schedules, (c) NCES standards, (d) performance cost controls, and (e) beneficial effort/method of performance criteria in fulfilling the contract. Education Department Directive C: GPA:2-105 dated 6/15/92 established the Standards and Guidelines for the Monitoring of Contracts.

In some instances, the contractor may request technical redirection for unanticipated problems. For simple matters that are clearly within the scope of the contract, such requests may be made verbally. For problems that may require a change in scope, all requests must be in writing and outline the issue(s) and potential options. The COR must use this information in discussions with other NCES senior management in determining the appropriate course of action. All changes in any contract scope of work require action by the Contracting Officer. Whatever course of action is taken, it must be documented and placed in the project files.

    GUIDELINE 3-3-1A: The COR should maintain close communication with the contractor. Depending on the nature of the survey, the COR should maintain communication through the use of meetings, phone calls, e-mails, visits, and/or the electronic management information system (MIS) for the purpose of tracking and monitoring the progress of the survey.

    GUIDELINE 3-3-1B: The COR should review and verify progress reports, vouchers, technical products and documentation, written correspondence, and other documents for the following purposes:

    1. Monitoring adherence to project schedules and requirements;
       
    2. Assuring deliverables meet NCES standards and comply with the conditions of the contract and other quality requirements (e.g., accuracy and completeness); and
       
    3. Identifying potential problems that would substantially affect the successful completion of the survey or alter the terms and conditions of the contract (e.g., cost or time increases, quality decreases).

    GUIDELINE 3-3-1C: The status of each unit of observation should be kept current and available to the COR at each stage of the data collection process. Critical status events may include, but are not limited to, dates of questionnaire mail out, returns, deletions (out-of-scopes), scan editing, data entry, machine editing, callback(s), and addition to the final data files. The COR should request direct and rapid access to the information.

    GUIDELINE 3-3-1D: To help decide whether any adjustments or corrective actions are needed, soon after initial startup of field operations, and less frequently thereafter, the COR should evaluate the quality of survey operations by comparing a sample of the original returned questionnaires with the information on the data file for the following purposes:

    1. Detect any data processing errors,
       
    2. Learn of any problems with reporting or questionnaire design, and
       
    3. Ensure that editing/update procedures are being correctly implemented.

    GUIDELINE 3-3-1E: On an as-needed basis, CORs may request a copy of "completed" records from the current master file (sometimes referred to as a "pull") and analyze the information for conformance to contract requirements. The extent of the statistical analysis of a pulled database should vary with survey objectives. Simple cross-tabulations and frequencies of discrete variables should normally point out internal coding inconsistencies and also provide interim item response rates. Simple descriptive statistics for continuous variables should provide interim item response rates, measures of dispersion, and outliers.

    GUIDELINE 3-3-1F: The COR should ensure that software used for weighting, imputations, and variance estimation is accurate. This may be done through a series of "checkpoints" imbedded within the program(s). Another alternative is to have the contractor provide printouts from a series of discrete steps with review by the COR along the way.

    GUIDELINE 3-3-1G: The COR should keep the CO and NCES management informed of the result of reviews. As an integral part of this work, the COR should offer recommendations for solving any problems, acceptance of deliverables, performance awards, and approval or disapproval of any proposed changes.


STANDARD 3-3-2: The COR must maintain the following documents in the COR contract file: (a) progress reports, (b) vouchers, and (c) deliverables as required by the contract. Together with the RFP, contract proposal, proposal evaluation, and signed contract, these documents are subject to audit. Also document any modifications or changes in (a) key personnel, (b) project schedule, (c) deliverables, and (d) scope of work, and their implications for the project completion date, deliverables, and costs.

    GUIDELINE 3-3-2A: It is advisable to include in the contract file all correspondence, such as logs of phone conversations, e-mail and written correspondence, and documentation, describing the approval of or decisions made regarding changes.

    GUIDELINE 3-3-2B: The COR should keep accurate and complete records of contractor performance, such as lateness, unacceptable deliverables, and cost overrun. Actions or decisions taken by the COR or CO to remedy the problems should also be clearly documented.


STANDARD 3-3-3: CORs should require that all computer programs (software) be self-documenting.

    GUIDELINE 3-3-3A: The programmer should insert "comments" within the program(s) to describe each discrete section of code. Relationships between programs and data files should be flowcharted or described in a separate document. This includes record layouts and file structures.


STANDARD 3-3-4: Upon completion and/or termination of the contract, the COR must archive those items specified in the Standard for Documenting a Survey System (3-4) and Standard for Survey Documentation in Center Reports (7-2).