SUBJECT: INITIAL PLANNING OF SURVEYS
NCES STANDARD: 1-1
PURPOSE: To provide an initial planning document that includes the information
required for a decision on whether or not to proceed with the preliminary design
and implementation plans of a specific survey or survey system.
KEY TERMS: assessment,
design effect, effect size,
effective sample size, key variables,
minimum substantively significant effect (MSSE), planning document, power,
response rate, and
STANDARD 1-1-1: The initial plan for developing a survey or survey system
must include the justification for the study and must describe the survey methodology.
Prior to an OMB fiscal year budget request for data collection, the initial
planning document must be presented to the OC/ODC for review and a decision
on whether to proceed with the design phase. The initial planning document must
include the following:
- A justification for the survey, including: the rationale for the survey,
the goals and objectives, and related hypotheses to be tested. This justification
must include evidence that consultations with potential users have occurred.
- A review of related studies, surveys, and reports of federal and non-federal
sources to ensure that part or all of the data are not available from an existing
source, or could not be more appropriately obtained by adding questions to existing
surveys sponsored by NCES or other agencies. The goal here is to minimize respondent
burden. If a new survey is needed, efforts should be made in the development
of the questionnaire and any assessment items to minimize the burden to individual
- Surveys that involve interviewing students in elementary and secondary schools
must adhere to the requirements of the Protection of Pupil Rights Act and related
amendments (see 20 US Code Section 1232h and amendments included in Section
1061 of the No Child Left Behind Act of 2002). Specifically, without written
consent from a student's parent, questions may not be asked about the following:
- Political affiliations or beliefs of the student or the student's parent;
- Mental or psychological problems of the student or the student's family;
- Sex behavior or attitudes;
- Illegal, anti-social, self-incriminating, or demeaning behavior;
- Critical appraisals of other individuals with whom respondents have close
- Legally recognized privileged or analogous relationships, such as those of
lawyers, physicians, and ministers;
- Religious practices, affiliations, or beliefs of the student or the student's
- Income (other than that required by law to determine eligibility for participation
or for receiving financial assistance under such a program).
In addition, the confidentiality and privacy provisions of the Privacy Act
and the Education Sciences Reform Act of 2002, must be taken
into account in designing any studies that will collect individually identifiable data
from any survey participants (see Standard 4-2).
- A preliminary survey design that discusses the proposed target population,
response rate goals (see Standard 2-3), sample design, sample size and effective sample size determination
based on power analyses for the MSSEs for key variables, data collection methods,
and methodological issues.
- A preliminary analysis plan that identifies analysis issues, objectives,
key variables, minimum substantively significant effect sizes, and proposed
- A list of data items that will be maintained over time as part of an NCES
data series, including the justification for each item.
- A preliminary time schedule that accounts for the complete survey cycle from
planning to data release.
- A preliminary publication and dissemination plan that identifies proposed
major publications and their target audiences (see Standard 1-2).
- A preliminary survey evaluation plan that identifies the proposed analyses
necessary for data users to understand the quality and limitations of the survey
(see Standard 4-3).
- An internal cost estimate that reflects all of the above items.