Appendix A: Technical Notes

Fast Response Survey System

The Fast Response Survey System (FRSS) was established in 1975 by the National Center for Education Statistics (NCES), U.S. Department of Education. FRSS is designed to collect issue-oriented data within a relatively short time frame. FRSS collects data from state education agencies, local education agencies, public and private elementary and secondary schools, public school teachers, and public libraries. To ensure minimal burden on respondents, the surveys are generally limited to three pages of questions, with a response burden of about 30 minutes per respondent. Sample sizes are relatively small (usually about 1,000 to 1,500 respondents per survey) so that data collection can be completed quickly. Data are weighted to produce national estimates of the sampled education sector. The sample size permits limited breakouts by classification variables. However, as the number of categories within the classification variables increases, the sample size within categories decreases, which results in larger sampling errors for the breakouts by classification variables.

Sample Design

The sample for the FRSS survey on foods and physical activity consisted of 1,198 regular public elementary schools in the 50 states and the District of Columbia. It was selected from the 2002-03 NCES Common Core of Data (CCD) Public School Universe file, which was the most current file available at the time of selection. The sampling frame included 50,980 regular elementary schools. For the purposes of the study, an elementary school was defined as a school with a high grade of 1 to 8 and a low grade of prekindergarten, kindergarten, or grades 1 to 3. Excluded from the sampling frame were schools with a high grade of prekindergarten or kindergarten and ungraded schools, along with special education, vocational, and alternative/other schools, schools outside the 50 states and the District of Columbia, and schools with zero or missing enrollment.

The public school sampling frame was stratified by enrollment size (less than 300, 300 to 499, 500 to 599, 600 to 749, and 750 or more) and percent eligible for free or reduced-price lunch (less than 35 percent, 36 to 49 percent, 50 to 74 percent, and 75 percent or more). Schools in the frame were then sorted by type of locale (city, urban fringe, town, and rural) and region (Northeast, Southeast, Central, and West) to induce additional implicit stratification. These variables are defined in more detail in the "Definitions of Analysis Variables" section of these Technical Notes.

Data Collection and Response Rates

Questionnaires and cover letters for the study were mailed to the principal of each sampled school in early March 2005. The letter introduced the study and requested that the questionnaire be completed by the person most knowledgeable about the availability of foods and opportunities for physical activity at the school. Respondents were encouraged to consult with the school's food service personnel and physical education staff to complete relevant sections of the questionnaire, as necessary. Respondents were also offered the option of completing the survey via the Web. The cover letter for the study included information on how to access the survey on the Web, including the survey Uniform Resource Location (URL) and the user login and password. Telephone follow-up for survey nonresponse and data clarification was initiated in late March 2005 and completed in late June 2005.

Of the 1,198 schools in the sample, 37 were found to be ineligible for the survey because they were closed or did not meet the grade requirements for inclusion as an elementary school. This left a total of 1,161 eligible schools in the sample. Completed questionnaires were received from 1,055 schools, or 91 percent of the eligible schools (table A-1). Of the schools that completed the survey, 19 percent completed it by Web, 53 percent completed it by mail, 27 percent completed it by fax, and 1 percent completed it by telephone.

The weighted response rate was 91 percent. The weighted number of eligible institutions in the survey represents the estimated universe of regular elementary schools in the 50 states and the District of Columbia. The estimated number of schools in the survey universe decreased from the 50,980 schools in the CCD sampling frame to an estimated 49,393 because some of the schools were determined to be ineligible for the FRSS survey during data collection.

Imputation for Item Nonresponse

Although item nonresponse for key items was very low, missing data were imputed for the 32 items with a response rate of less than 100 percent (table A-2).¹ The missing items included both numerical data such as total minutes per day of scheduled recess, as well as categorical data such as whether soft drinks were available at vending machines. The missing data were imputed using a "hot-deck" approach to obtain a "donor" school from which the imputed values were derived. Under the hot-deck approach, a donor school that matched selected characteristics of the school with missing data (the recipient school) was identified. The matching characteristics included enrollment size, percent of students in the school eligible for free or reduced-price lunch, and type of locale. In addition, relevant questionnaire items were used to form appropriate imputation groupings. Once a donor was found, it was used to obtain the imputed values for the school with missing data. For both categorical and numerical items, the imputed value was simply the corresponding value from the donor school. All missing items for a given school were imputed from the same donor.

Data Reliability

While the Foods and Physical Activity survey was designed to account for sampling error and to minimize nonsampling error, estimates produced from the data collected are subject to both types of error. Sampling error occurs because the data are collected from a sample rather than a census of the population, and nonsampling errors are errors made during the collection and processing of the data.

Sampling Errors

The responses were weighted to produce national estimates (table A-1). The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability. General sampling theory was used to estimate the sampling variability of the estimates and to test for statistically significant differences between estimates.

The standard error is a measure of the variability of an estimate due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of public elementary schools that sold foods to generate funds is 36.3 percent, and the standard error is 1.5 percent (tables 2 and 2a). The 95 percent confidence interval for the statistic extends from [36.3 - (1.5 x 1.96)] to [36.3 + (1.5 x 1.96)], or from 33.4 to 39.2 percent. The 1.96 is the critical value for a statistical test at the 0.05 significance level (where 0.05 indicates the 5 percent of all possible samples that would be outside the range of the confidence interval).

Because the data from the FRSS foods and physical activity survey were collected using a complex sampling design, the variances of the estimates from this survey (e.g., estimates of proportions) are typically different from what would be expected from data collected with a simple random sample. Not taking the complex sample design into account can lead to an underestimation of the standard errors associated with such estimates. To generate accurate standard errors for the estimates in this report, standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. To construct the replications, 50 stratified subsamples of the full sample were created and then dropped 1 at a time to define 50 jackknife replicates. A computer program (WesVar) was used to calculate the estimates of standard errors. WesVar is a stand-alone Windows application that computes sampling errors from complex samples for a wide variety of statistics (totals, percents, ratios, log-odds ratios, general functions of estimates in tables, linear regression parameters, and logistic regression parameters).

Nonsampling Errors

Nonsampling error is the term used to describe variations in the estimates that may be caused by population coverage limitations and data collection, processing, and reporting procedures. The sources of nonsampling errors are typically problems like unit and item nonresponse, differences in respondents' interpretations of the meaning of questions, response differences related to the particular time the survey was conducted, and mistakes made during data preparation. It is difficult to identify and estimate either the amount of nonsampling error or the bias caused by this error. To minimize the potential for nonsampling error, this study used a variety of procedures, including a pretest of the questionnaire with principals of elementary schools. The pretest provided the opportunity to check for consistency of interpretation of questions and definitions and to eliminate ambiguous items. The questionnaire and instructions were also extensively reviewed by NCES. In addition, manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone to resolve problems. Data were keyed with 100 percent verification for surveys received by mail, fax, or telephone.

Definitions of Analysis Variables

Many of the school characteristics, described below, may be related to each other. For example, school enrollment size and locale are related, with city schools typically being larger than rural schools. Other relationships between these analysis variables may exist. However, this E.D. TAB report focuses on bivariate relationships between the analysis variables and questionnaire variables rather than more complex analyses.

Enrollment Size - This variable indicates the total number of students enrolled in the school based on data from the 2002-03 CCD. The variable was collapsed into the following three categories:

• Less than 300 students (small)
• 300 to 499 students (medium)
• 500 or more students (large)

School Locale - This variable indicates the type of community in which the school is located, as defined in the 2002-03 CCD (which uses definitions based on U.S. Census Bureau classifications). This variable was based on the eight-category locale variable from CCD, recoded into a four-category analysis variable for this report. Large and midsize cities were coded as city, the urban fringes of large and midsize cities were coded as urban fringe, large and small towns were coded as town, and rural areas outside and inside Metropolitan Statistical Areas (MSAs) were coded as rural. The categories are described in more detail below.

• City - A large or midsize central city of a Consolidated Metropolitan Statistical Area (CMSA) or Metropolitan Statistical Area (MSA).
• Urban fringe - Any incorporated place, Census-designated place, or non-place territory within a CSMA or MSA of a large or midsize city, and defined as urban by the Census Bureau.
• Town - Any incorporated place or Census-designated place with a population greater than or equal to 2,500 and located outside a CMSA or MSA.
• Rural - Any incorporated place, Census-designated place, or non-place territory defined as rural by the Census Bureau.

Region - This variable classifies schools into one of the four geographic regions used by the Bureau of Economic Analysis of the U.S. Department of Commerce, the National Assessment of Educational Progress, and the National Education Association. Data were obtained from the 2002-03 CCD School Universe file. The geographic regions are:

• Northeast - Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont
• Southeast - Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia
• Central - Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin
• West - Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming

Percent Minority Enrollment - This variable indicates the percentage of students enrolled in the school whose race or ethnicity is classified as one of the following: American Indian or Alaska Native, Asian or Pacific Islander, non-Hispanic Black, or Hispanic, based on data in the 2002-03 CCD School Universe file. Data on this variable were missing for 21 schools; schools with missing data were excluded from all analyses by percent minority enrollment. The percent minority enrollment variable was collapsed into the following four categories:

• Less than 6 percent minority
• 6 to 20 percent minority
• 21 to 49 percent minority
• 50 percent or more minority

Percent of Students Eligible for Free or Reduced-Price Lunch-This variable was based on responses to question 16 on the survey questionnaire; if it was missing from the questionnaire (3.9 percent of all cases), it was obtained from the 2002-03 CCD School Universe File. This item served as a measurement of the concentration of poverty at the school. The categories are:

• Less than 35 percent
• 35 to 49 percent
• 50 to 74 percent
• 75 percent or more

Contact Information

For more information about the survey, contact Bernie Greene, Early Childhood, International, and Crosscutting Studies Division, National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, 1990 K Street NW, Washington, DC 20006; telephone (202) 502-7348.

Top

---