SUBJECT: DESIGN OF SURVEYS

NCES STANDARD: 2-1

PURPOSE: To identify the survey design components required to conduct a data collection.

KEY TERMS: confidentiality, domain, effective sample size, estimation, field test, frame, individually identifiable data, key variables, planning document, precision, probability of selection, response rates, strata, survey, survey system, target population, and variance.

STANDARD 2-1-1: A technical document that delineates the basic design of a survey or survey system must be developed prior to the initiation of a data collection. The document must address: the objectives of the survey as indicated in the initial planning document; the survey design; the data collection plan; and the personnel resources, funds, and time needed to achieve high data quality. To meet this standard, the survey design plan must include the following:

1. A detailed discussion of the goals and objectives of the survey or survey system, including the information needs that will be met, content areas included, target population(s), and analytic goals.
    
2. A discussion of the sample design that describes how it will yield the data required to meet the objectives of the survey. The discussion must include the following: identification of the sampling frame and the adequacy of the frame (see Standard 3-1); sampling strata; power analyses to determine sample sizes and effective sample size for key variables by reporting domains, sample size by stratum; the known probability of selection; expected yield by stratum; estimated efficiency of sample design; weighting plan; variance estimation techniques appropriate to the survey design; and expected precision of estimates for key variables.
    
3. A listing of all survey data items, including time series data items, how each item can best be measured (e.g., through questionnaires, tests), and reasonable evidence that these items are valid and can be measured both accurately and reliably.
    
4. An analysis plan providing evidence that the basic information needs which justify the study can be met through the proposed data collection. The plan must demonstrate how the proposed sample, the survey items, and the measurement methods are related to the objectives of the survey.
    
5. The anticipated data collection procedures including: timing of data collection; primary mode of collection; and methods for achieving acceptable response rates (see Standard 3-2).
    
6. A plan for preserving the confidentiality of the data during collection, processing, and analysis, if individually identifiable data will be collected. An analysis plan for disclosure risk control is also required to prepare a public use data file (see Standard 4-2).
    
7. An outline of a plan for quality assurance during each phase of the survey process that will permit monitoring and assessing the performance during implementation. The plan must include contingencies to modify the survey procedures, if design parameters appear unlikely to meet expectations (for example, low response rates) (see Standard 3-3).
    
8. A plan for field testing the survey or survey system (see Standard 2-4).
    
9. An outline of the general parameters for evaluating survey procedures and results (see Standard 4-3).
    
10. General specifications for an internal project management system that identifies critical activities and key milestones of the survey that will be monitored, and the time relationships among them (see Standard 3-3).
     
11. An Independent Government Cost Estimate (IGCE) for the entire study, including, for example, the pilot test, the main study, file preparation and documentation, disclosure risk analysis, the survey evaluation, and analysis and reporting.