Appendix A — Survey Methodology and Data Reliability

Sample Selection

The sampling frame of schools for the FRSS nutrition education teacher survey was constructed from the 1993-94 NCES Common Core of Data (CCD) public school universe file and included over 61,000 regular elementary schools. For the purposes of the survey, elementary schools were defined to be those with a beginning grade of sixth or lower and no grade higher than eighth. Excluded from the frame were special education, vocational, and alternative/other schools, schools in the U.S. territories, and schools with a highest grade lower than grade one.

Samples were selected in two stages, first elementary schools, and then teachers within the sampled schools. The CCD frame was stratified by locale (city, urban fringe, town, rural), crossed by enrollment size (less than 300, 300 to 499, and 500 or more). Within each primary stratum, schools were sorted by geographic region (Northeast, Southeast, Central, West) and a measure of poverty status (based on the percentage of students eligible for free or reduced-price lunch) prior to sample selection to induce additional implicit stratification. A sample of 750 schools was then selected from the sorted frame with probabilities proportionate to size (PPS), where the measure of size was the estimated number of full-time-equivalent (FTE) teachers in the school. It should be noted that FTE teacher counts were missing for about 2 percent of the schools in the CCD file. For these, the required measure of size was imputed by applying the average enrollment-to- FTE teacher ratio for schools in the same locale and enrollment size class category to the enrollment of the school with the missing FTE teacher count.

To facilitate the selection of teachers, each sampled school was requested to provide a comprehensive list of their teachers of grades kindergarten through five who taught self-contained classes. Lists were obtained from 96 percent of the selected elementary schools, yielding 705 participating schools. An average of about two elementary school teachers (fewer for schools with a smaller number of eligible teachers and more for schools with a larger number of eligible teaches) was then selected from each participating school, for a total initial sample size of 1,409.

Respondents and Response Rates

In February 1997, questionnaires (see appendix C) were mailed to 1,409 public elementary school teachers. Of the teachers sampled, 62 were found to be out of scope (no longer at the school, or not assigned to a selfcontained class), leaving 1,347 eligible teachers in the sample. Telephone followup was initiated in March, and data collection was completed on July 3, with 1,180 respondents. The teacher response rate was 88 percent. This figure combines with the response rate from the list collection for a final response rate of 84 percent. Item nonresponse rates ranged from 0.0 to 1.0 percent.

Sampling and Nonsampling Errors

The responses were weighted to produce national estimates. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in the collection of the data. These errors can sometimes bias the data. Nonsampling errors may include such problems as the differences in the respondents' interpretations of the meaning of the questions; memory effects; misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

To minimize the potential for nonsampling errors, the questionnaire was pretested with knowledgeable respondents like those who completed the survey. During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous terms. The questionnaire and instructions were extensively reviewed by the Food and Nutrition Service and the National Center for Education Statistics. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Imputations for item nonresponse were not implemented, as item nonresponse rates were very low. Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of an estimate due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is what is referred to as a 95 percent confidence interval. For example, the estimated percentage of teachers who taught about nutrition in school year 1996-97 is 88 percent and the estimated standard error is 1.1 percent. The 95 percent confidence interval for this statistic extends from 88 - (1.96 * 1.1) to 88 + (1.96 * 1.1), or from 85.8 to 90.2.

Estimates of standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. To construct the replicates, 50 stratified subsamples of the full sample were created and then dropped, one at a time, to define 50 jackknife replicates. A proprietary computer program (WESVAR), available at Westat, was used to calculate the estimates of standard errors.

Background Information

The survey was conducted under contract with Westat, using the NCES Fast Response Survey System (FRSS). Westat's project director was Elizabeth Farris, and the survey manager was Carin Celebuski. Shelley Burns and Judi Carpenter were the NCES project officers. The data were requested by Leslie Christovich and Marie Mitchell from the Food and Nutrition Service of the USDA. Marie Mitchell coordinated the project for the USDA. The survey instrument was developed with input from several persons in the field of nutrition education, including individuals from the USDA's Nutrition and Technical Services Division and Team Nutrition. The report was reviewed by the following individuals:

Outside NCES

• Marie Mitchell, CDC



• Leslie Lytle, University of Minnesota

Inside NCES

• Marilyn McMillen



• Kathryn Chandler



• Larry Bobbitt



• Jonaki Bose

For more information about the Fast Response Survey System or the nutrition education teacher survey, contact Shelley Burns, Elementary/ Secondary Statistics Division, Office of Educational Research and Improvement, National Center for Education Statistics, 555 New Jersey Avenue, NW, Washington, DC 20208-5651, telephone (202) 219-1463.

Terms Defined on the Survey Questionnaire

Collaborative or cooperative work- students work together in small groups to solve problems or do projects.

Coordinated school nutrition policy- may address such issues as coordinating nutrition education across subjects and across grades, collaboration between the school meals program and the classroom, and policies on outside food vendors and closed lunch periods.

Coordinating nutrition education across subjects and across grades- refers to the integration of nutrition lessons into subjects such as math and science, and the integration across grades so the lessons at each grade level build on the previous year's lessons.

Hands-on learning- students engage in direct learning experiences by applying their learning to real-life situations or everyday issues and events.

Nutrition education materials- lesson plans, curriculum guides, posters, pamphlets, multimedia, etc., designed to improve health, achieve positive change in dietary habits, and emphasize the relationship between diet and health.

Nutrition education- refers to curricula, courses, lesson plans and units, and activities designed to provide instruction with regard to the nutritional value of foods and the relationship between food and human health.

Classification Variables

Instructional level of the teacher
Kindergarten through second-grade teachers.
Third through fifth-grade teachers.

School enrollment size
Less than 300 students in the school.
Between 300-499 students in the school.
500 or more students in the school.

Geographic region of the school
Northeast - Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania,
Rhode Island, and Vermont.
Southeast - Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.
Central - Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin.
West - Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming.

Level of support for nutrition education at the school - constructed from information reported by teachers on their questionnaires (questions 2 and 3). Affirmative responses to six questions asking about the availability of specific resources and policies in support of nutrition education at the school were summed. The six resources were ongoing inservice training that focuses on teaching strategies for behavioral change; school food service personnel serving healthy, well-balanced meals in the cafeteria; reference materials on nutrition education available at your school; support from your school or district for nutrition education as a valid use of instructional time; a written policy or guidelines on nutrition education from your school, district, or state; and a coordinated school nutrition policy.
Low - teachers reporting 0-3 resources available to them.
High - teachers reporting 4-6 resources available to them.

Type of nutrition education training the teacher has received - constructed from information reported by teachers on their questionnaires (question 17). Teachers were recoded to their most formal type of training, although they may have participated in other types of training as well.
None - teachers reported not participating in any training for teaching students about nutrition.
Research on own - teachers reported that they did research on their own, but did not participate in any formal training.
Inservice - teachers reported that they participated in inservice or other professional development training, but did not participate in training as an undergraduate or graduate student.
College coursework - teachers reported that they participated in training as an undergraduate or graduate student.

Top