SUBJECT: DESIGN OF SURVEYS
NCES STANDARD: 2-1
PURPOSE: To identify the survey design components required to conduct
a data collection.
KEY TERMS: confidentiality,
domain, effective sample size, estimation,
individually identifiable data,
planning document, precision,
probability of selection, response rates,
strata, survey, survey system,
STANDARD 2-1-1: A technical document that delineates the basic design
of a survey or survey system must be developed prior to the initiation of a
data collection. The document must address: the objectives of the survey as
indicated in the initial planning document; the survey design; the data collection
plan; and the personnel resources, funds, and time needed to achieve high data
quality. To meet this standard, the survey design plan must include the following:
- A detailed discussion of the goals and objectives of the survey or survey system, including the information needs that will be met, content areas included,
target population(s), and analytic goals.
- A discussion of the sample design that describes how it will yield the data
required to meet the objectives of the survey. The discussion must include the
following: identification of the sampling frame
and the adequacy of the frame
(see Standard 3-1); sampling strata; power analyses to determine sample sizes and effective sample size
for key variables by reporting domains, sample size by stratum; the known probability of selection; expected yield by stratum; estimated efficiency of sample design;
weighting plan; variance estimation techniques appropriate to the survey design;
and expected precision
of estimates for key variables.
- A listing of all survey data items, including time series data items, how
each item can best be measured (e.g., through questionnaires, tests), and reasonable
evidence that these items are valid and can be measured both accurately and
- An analysis plan providing evidence that the basic information needs which
justify the study can be met through the proposed data collection. The plan
must demonstrate how the proposed sample, the survey items, and the measurement
methods are related to the objectives of the survey.
- The anticipated data collection procedures including: timing of data collection;
primary mode of collection; and methods for achieving acceptable response rates
(see Standard 3-2).
- A plan for preserving the confidentiality of the data during collection,
processing, and analysis, if individually identifiable data will be collected.
An analysis plan for disclosure risk control is also required to prepare a public
use data file (see Standard 4-2).
- An outline of a plan for quality assurance during each phase of the survey
process that will permit monitoring and assessing the performance during implementation.
The plan must include contingencies to modify the survey procedures, if design
parameters appear unlikely to meet expectations (for example, low response rates)
(see Standard 3-3).
- A plan for field testing the survey or
survey system (see Standard 2-4).
- An outline of the general parameters for evaluating survey procedures and
results (see Standard 4-3).
- General specifications for an internal project management system that identifies
critical activities and key milestones of the survey that will be monitored,
and the time relationships among them (see Standard 3-3).
- An Independent Government Cost Estimate (IGCE) for the entire study, including,
for example, the pilot test, the main study, file preparation and documentation,
disclosure risk analysis,
the survey evaluation, and analysis and reporting.