Appendix A—Survey Methodology and Data Reliability

The sampling frame for the FRSS Nutrition Education Survey was constructed from the 1992-93 NCES Common Core of Data (CCD) public school universe file and included over 78,000 public elementary, middle, and high schools. Excluded from the frame were special education, vocational, and alternative/other schools, schools in the territories, and schools with the highest grade lower than grade one.

Sample Selection

Separate samples of 333 elementary, 333 middle, and 334 high schools were selected for the survey. The samples were stratified by geographic region (northeast, southeast, central, west), metropolitan status (city, urban fringe, town, rural), and school size (less than 300; 300 to 499; and 500 or more) (table A-1).

Respondents and Response Rates

In April of 1995, questionnaires (see appendix C) were mailed to 1,000 public school principals. The principals were asked to direct the survey to the person most knowledgeable about nutrition education in the school and ask that person to complete the survey. Of the schools sampled, 12 were found to be out of scope (no longer at the same location or serving the same population), leaving 988 eligible schools in the sample. Telephone followup was initiated in mid-May and data collection was completed on July 7, with 916 respondents. The final response rate was 93 percent. Item nonresponse rates ranged from 0.0 percent to 1.0 percent.

Sampling and Nonsampling Errors

The responses were weighted to produce national estimates. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in the collection of the data. These errors can sometimes bias the data. Nonsampling errors may include such problems as the differences in the respondents' interpretations of the meaning of the questions; memory effects; misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

To minimize the potential for nonsampling errors, the questionnaire was pretested with knowledgeable respondents like those who completed the survey. During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous terms. The questionnaire and instructions were extensively reviewed by the Food and Consumer Service and the National Center for Education Statistics. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Imputations for item nonresponse were not implemented, as item nonresponse rates were very low. Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size.

Standard errors are used as a measure of the precision expected from a particular sample. If all possible sampled were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is what is called a 95 percent confidence interval. For example, the estimated percentage of public elementary schools with no nutrition education coordination is 66 percent and the estimated standard error is 3.0 percent. The 95 percent confidence interval for this statistic extends from [66 - (1.96 x 3) to 66 + (1.96 x 3)], or from 61.5 to 70.5.

Estimates of standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involved constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. To construct the replications, 50 stratified subsamples of the full sample were created and then dropped, one at a time, to define 50 jackknife replicates. A proprietary computer program (WESVAR), available at Westat, Inc., was used to calculate the estimates of standard errors.

Background Information

The survey was conducted under contract with Westat, Inc., using the NCES Fast Response Survey System (FRSS). Westat's project director was Elizabeth Farris, and the survey manager was Carin Celebuski. Judi Carpenter was the NCES project officer. The data were requested by Leslie Christovich and Marie Mitchell from the Food and Consumer Service of the USDA. Marie Mitchell coordinated the project for the USDA.

The survey instrument was developed with input from several persons in the field of nutrition education, including individuals from the Nutrition Education and Training Program and the Education Information Advisory Committee (EIAC) of the Council of Chief State School Officers. The EIAC committee members were the following:

• John Perkins, Texas
• Kathy Kuser, New Jersey
• Joe Worden, Florida
• Josephine Busha, Vermont
• Betty Marcelynas, Washington
• Jim Burke, Illinois
• Maria Balakshin, California
• Michael Smith, Wyoming

The report was reviewed by the following individuals:

Outside NCES

• Leslie Lytle, University of Minnesota
• Barbara Shannon, Pennsylvania State University

Inside NCES

• Jonaki Bose, NCES
• Mike Cohen, NCES
• Edith McArthur
• Mary Rollefson, NCES
• Peter Stowe, NCES

For more information about the Fast Response Survey System or the Nutrition Education Survey, contact Judi Carpenter, Elementary/Secondary Statistics Division, Office of Educational Research and Improvement, National Center for Education Statistics.

Terms Defined on the Survey Questionnaire

Nutrition education - refers to curricula, courses, lesson plans and units, and activities designed to provide instruction with regard to the nutritional value of foods and the relationship between food and human health. Nutrition education can also be provided through nonclassroom activities and events such as special assemblies and health fairs, etc.

School health program - a program that may include school health education and physical education; school-linked or school-based health services designed to prevent, detect, and address health problems; psychological assessment and counseling to promote child development and emotional health; healthful school food service selections; schoolsite health promotion for faculty and staff; and integrated school and community health promotion efforts.

Nutrients - the nourishing components in food, such as vitamins, minerals, proteins, carbohydrates, fats, etc.

Nutrition-related area - academic subject areas related to nutrition (e.g., home economics, science, health, physical education, dietetics).

Cooperative Extension Service - a public-funded, nonformal educational system that links the USDA, land-grant universities, and counties. Its purpose is to diffuse practical information on subjects related to agriculture, home economics, and rural energy.

Materials - lesson plans, curriculum guides, posters, pamphlets, and multimedia, etc., designed to improve health, achieve positive change in dietary habits, and emphasize the relationship between diet and health.

Classification Variables

Metropolitan Status

Urban - a central city of a Metropolitan Statistical Area (MSA).

Urban fringe - a place within an MSA, but not primarily its central city.

Town - a place not within an MSA, but with a population greater than or equal to 2,500, and defined as urban by the U.S. Bureau of the Census.

Rural - a place with a population less than 2,500 and defined as rural by the U.S. Bureau of the Census.

Geographic Region

Northeast - Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.

Southeast - Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.

Central - Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin.

West - Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming.

Instructional Level

Elementary school - Schools beginning with grade 6 or lower and with no grade higher than 8.

Middle school - Schools with a low grade of 4 to 7 and a high grade of 4 to 9.

High school - Schools with a grade 12 and no grade lower than 9. Combined schools, with both elementary and secondary grades, were included with high schools for sampling purposes.

Top