Appendix A—Survey Methodology

Fast Response Survey System

The Fast Response Survey System (FRSS) was established in 1975 by the National Center for Education Statistics (NCES), U.S. Department of Education. FRSS is designed to collect issue-oriented data within a relatively short timeframe. To ensure minimal burden on respondents, the surveys are generally limited to three pages of questions, with a response burden of about 30 minutes per respondent. Sample sizes are relatively small (usually about 1,000 to 1,500 respondents per survey) so that data collection can be completed quickly. Data are weighted to produce national estimates of the sampled education sector. The sample size permits limited breakouts by classification variables. However, as the number of categories within the classification variables increases, the sample size within categories decreases, which results in larger sampling errors for the breakouts by classification variables. FRSS collects data from state education agencies, local education agencies, public and private elementary and secondary schools, public school teachers, and public libraries.

Sample Selection

The sample for the FRSS Survey of Classes That Serve Children Prior to Kindergarten in Public Schools consisted of 2,044 regular and special education public schools in the 50 states and the District of Columbia. The sample contained 1,801 regular elementary schools, 150 regular combined schools, and 93 special education schools. It was selected from the 1998–99 NCES Common Core of Data (CCD) School Universe file (see Table A - 1). Regular middle and high schools were not eligible for the survey. Also excluded from the sampling frame were vocational and alternative/other schools, schools in the territories, and Department of Defense and Bureau of Indian Affairs schools.

A school was defined as an elementary school if the lowest grade was less than or equal to grade 3 and the highest grade was less than or equal to grade 8. A middle school was defined as having a low grade of 4 or more and a high grade of 9 or less. A high school was defined as having a low grade of 9 or more and a high grade of 10 or more. Combined schools contain both elementary and secondary grades (e.g., K to 12 or 1 to 9).

The sample design was guided by the study's focus on classes that serve children prior to kindergarten entry in public schools. The 1998–99 CCD frame indicates that of the nation's 85,000 regular and special education schools, approximately one-third of the elementary and special education schools reported having either prekindergarten children or a prekindergarten grade (Table A - 1). By definition, middle and high schools do not offer elementary grades and are therefore not expected to offer prekindergarten classes. The 1998–99 CCD indicates that very few middle and secondary schools reported prekindergarten children or grade. Although this does not necessarily imply that the "true" incidence of prekindergarten classes in these schools is 0, it does seem likely that the incidence of such classes is very low. Moreover, information from a variety of sources (e.g., pretests, feasibility calls, pilot study) suggests it is highly likely that prekindergarten classes in middle and high schools were established to serve the needs of older students (e.g., as laboratories for students' on-the-job-training or as day care for the children of high school students). Thus, only the approximately 51,000 elementary schools, 5,000 combined schools, and 2,000 special education schools were included in the frame for the survey.

The sample design was also informed by results from a pilot study of over 300 schools that was conducted in November 2000 to obtain relevant information about the presence and nature of prekindergarten classes in public schools. The pilot study was necessary because the information in the CCD file underreported prekindergarten enrollment; for example, some states, such as California, Alabama, and Kentucky, did not report any prekindergarten classes or students to CCD. Feasibility calls and pretests suggested several reasons for the underreporting of prekindergarten children to CCD. For example, some schools did not refer to their classes for children preceding kindergarten as prekindergarten classes, and other schools did not report prekindergarten children if the prekindergarten class funds were kept separately from school funds. For these and other reasons, it was believed that a portion of the schools that reported no prekindergarten children or grade in CCD actually had some type of program for children prior to kindergarten entry.

The classes include general prekindergarten/ preschool, special education, Title I, Head Start administered by the school district, and any other classes that serve children prior to kindergarten entry, regardless of whether the schools referred to those classes as prekindergarten or preschool or some other name. Thus, the pilot was designed to (1) identify definitional issues and develop a working definition of prekindergarten, and (2) inform the study's sample design by answering What percentage of public schools listed in CCD as not having prekindergarten classes actually have classes that serve children prior to kindergarten entry?

The pilot study indicated that information available in the CCD file about the presence of students/classes prior to kindergarten was imperfect and could not be used to exclude schools without such programs from the main study sample. As a result, to avoid bias associated with undercoverage, it was necessary to include in the sample schools that reported no prekindergarten students/classes in CCD.

The 1998–99 CCD School Universe file contained information on approximately 58,000 potentially eligible public elementary schools. According to that file, 31 percent (about 18,000) of the schools had prekindergarten children/grades; 69 percent (about 40,000) of the schools did not have such children/grades. Based on pilot study findings, however:

• An estimated 20 percent of schools that CCD indicated as having prekindergarten classes (about 3,600 of the 18,000) did not offer such classes
   
• An estimated 19 percent that CCD indicated as not having prekindergarten classes (about 5,700 of the 40,000) did offer such classes.
   

Overall, the pilot study results estimated that 38 percent of the 58,000 potentially eligible public schools had classes for children prior to kindergarten entry, regardless of whether the schools referred to those classes as prekindergarten or preschool or some other name.⁴²For the main study, a stratified sample of 2,044 schools was selected from the 1998–99 CCD School Universe file. Information from the pilot study was used to guide the allocation of the total sample to various subsets of schools—schools with one or more prekindergarten children, schools with a prekindergarten grade but no prekindergarten children, and schools with no prekindergarten children or grade. Within each subset, schools were further stratified by poverty concentration (based on the percentage of students who are eligible for free or reduced-price lunch) and enrollment size. Stratification by poverty concentration was designed to ensure that sufficient numbers of high-poverty schools were selected for analysis purposes. Within each poverty group, the sample was allocated to size strata in rough proportion to the aggregate square root of the enrollment in the stratum. Such an allocation was expected to yield relatively precise estimates of percentages (e.g., the percentage of schools with prekindergarten classes that have a specified characteristic), as well as aggregative measures related to prekindergarten enrollment (e.g., the number of classes or students enrolled in prekindergarten classes). Finally, schools in the sampling frame were sorted by type of locale (city, urban fringe, town, rural) and region (Northeast, Southeast, Central, West) to induce additional implicit stratification. Within each sampling stratum, schools were selected systematically and with equal probabilities.

Respondent and Response Rates

Questionnaires and cover letters for the study were mailed to the principal of each sampled school in early March 2001. The letter introduced the study and requested that the questionnaire be completed by the person at the school who was most knowledgeable about prekindergarten classes. Telephone follow up was conducted from late March 2001 through May 2001 with principals who did not respond to the initial questionnaire mailing.

Of the 2,044 schools in the sample, 50 were found to be out of the scope of the survey, usually because they were administrative centers or closed schools. This left a total of 1,994 eligible schools in the sample. Completed questionnaires were received from 1,843 schools, or 92 percent of the eligible schools (Table A - 2). The weighted response rate was 94 percent. Roughly 80 percent (1,593) of the eligible cases required telephone follow up to obtain their participation. Among the respondents, 41 percent completed the questionnaire by mail, 43 percent completed it by telephone, and 15 percent provided their answers by fax.

With the exception of the question on the number of prekindergarten children eligible for free and reduced-price lunch (which had an item nonresponse rate of 5.6 percent), weighted item nonresponse rates ranged from 0 percent to 1.2 percent.

The responses were weighted to produce national estimates (see Table A - 2). The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The probabilities of selection varied by type of school (e.g., regular vs. special education), enrollment size, and whether or not the school enrolled prekindergarten students as recorded in the CCD frame. Nonresponse adjustments were made to reflect differential response rates by poverty status, in addition to type and size of school, and presence of prekindergarten students.

Sampling and Nonsampling Errors

The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability. The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of public elementary schools with prekindergarten classes is 35.3 percent, and the estimated standard error is 0.96 percent. The 95 percent confidence interval for the statistic extends from [35.3 – (0.96 times 1.96)] to [35.3 + (0.96 times 1.96)], or from 33.4 to 37.2 percent. The coefficient of variation ("c.v.," also referred to as the "relative standard error") of an estimate (y) is defined as c.v. = (s.e. / y) x 100, where s.e. is the standard error of the estimate y. Throughout this report, any estimate with a c.v. higher than 50 percent is flagged with the note that the estimate should be interpreted with caution because the value of the estimate is very unstable.

Estimates of standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variances of the statistics. To construct the replications, 50 stratified subsamples of the full sample were created and then dropped one at a time to define 50 jackknife replicates. A computer program (WesVar) was used to calculate the estimates of standard errors.

The test statistics used in the analysis were calculated using the jackknife variances and thus appropriately reflected the complex nature of the sample design. Bonferroni adjustments were made to control for multiple comparisons where appropriate. For example, for an "experiment-wise" comparison involving g pairwise comparisons, each difference was tested at the 0.05/g significance level to control for the fact that g differences were simultaneously tested. The Bonferroni adjustment results in a more conservative critical value being used when judging statistical significance. This means that comparisons that would have been significant with an unadjusted critical value of 1.96 may not be significant with the more conservative critical value. For example, the critical value for comparisons between any two of the four categories of poverty concentration is 2.64 rather than 1.96.

When comparing percentage estimates across a family of three or more ordered categories (e.g., categories defined by percent minority enrollment), however, a trend analysis was used rather than a series of paired comparisons. The trend test involved estimating a simple linear regression model with the ordered categories as the independent variable and the (dichotomous) outcome of interest (e.g., whether or not the school offered prekindergarten classes) as the dependent variable. The slope of the line (regression coefficient) describing the relationship between the independent and dependent variables was estimated using generalized weighted least squares. The corresponding standard error was estimated using jackknife replication methods. The test statistic used to assess the significance of the linear model was calculated as the ratio of the estimated regression coefficient to its standard error. If t was greater than 1.96 (the critical value of t with "infinite" degrees of freedom at a significance level of 0.05), there was evidence of a linear relationship between the two variables.

Regression models were also used to test the significance of combinations of independent variables (e.g., school size and prekindergarten class schedule) on a reported characteristic of interest. In particular, logistic regression methods allow the estimation of the probability of an event (e.g., the provision of meals) as a function of a number of independent variables and their interactions. To account for the complex sample design used in the study, jackknife replication was used to estimate the standard errors of the regression coefficients and to develop the corresponding statistical tests. An independent variable in the model was deemed to be statistically significant if the p-value associated with the test was less than 0.05.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in data collection. These errors can sometimes bias the data. Nonsampling errors may include such problems as misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. Although general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

To minimize the potential for nonsampling errors, the questionnaire was pretested with respondents like those who completed the survey. During the design of the survey, the survey pretest, and the pilot study, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were extensively reviewed by the National Center for Education Statistics and the Early Childhood Institute, U.S. Department of Education.⁴³ Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Data were keyed with 100 percent verification.

Definitions of Analysis Variables

School size – total number of students enrolled in the school based on data from the 1998–99 CCD School Universe file.

Less than 300 students
300 to 599 students
600 or more students

Enrollment data were missing for eight schools. However, the information needed was obtained through telephone contact with the schools, or through accessing the school data contained in the 1997–98 CCD file. Once this information was obtained, the eight schools were assigned to the appropriate enrollment size category.

Locale – as defined in the 1998–99 CCD School Universe file.

City – a large or midsize central city of a Consolidated Metropolitan Statistical Area (CMSA) or Metropolitan Statistical Area (MSA).

Urban fringe/large town – urban fringe is a place within a CMSA or MSA of a large or midsize central city, but not primarily its central city; large town is an incorporated place not within a CSMA or MSA, with a population greater than or equal to 25,000.

Small town/rural – small town is an incorporated place not within a CMSA or MSA, with a population less than 25,000 and greater than or equal to 2,500; rural is a place either outside or within a CMSA or MSA of a large or midsize city, and defined as rural by the U.S. Bureau of the Census.

Geographic region – one of four regions used by the Bureau of Economic Analysis of the U.S. Department of Commerce, the National Assessment of Educational Progress, and the National Education Association. Obtained form the 1998–99 CCD School Universe file.

Northeast – Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.

Southeast – Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.

Central – Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin.

West – Alaska, Arizona, California , Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming.

Percent minority enrollment – The percent of students enrolled in the school whose race or ethnicity is classified as one of the following: American Indian or Alaska Native, Asian, Black, or Hispanic, based on data in the 1998–99 CCD School Universe file. The data were missing for 24 schools in the respondent sample.

Less than 6 percent
6 to 20 percent
21 to 50 percent
50 percent or more

Percent of students eligible for free or reduced-price lunch – This was based on information collected from the schools during the survey. The data were missing from 201 questionnaires. Data for 184 of these schools were obtained from the 1998–99 CCD School Universe file, leaving 17 schools for which the data were missing. This item served as the measurement of the concentration of poverty at the school.

Less than 35 percent
35 to 49 percent
50 to 74 percent
75 percent or more

Definitions of Questionnaire Terminology

Children who are limited English proficient (LEP) – Children whose native or dominant language is other than English, and whose skills in listening to, speaking, reading, or writing English are such that they derive little benefit from school instruction in English.

General education classes for children prior to kindergarten – Includes combined/inclusive classes, Title I classes, Head Start classes that were part of a district-administered program, and any other classes primarily for 3- or 4-year-olds prior to kindergarten.

Public elementary school – Refers to public special education and regular elementary and combined schools. A school was defined as an elementary school if the lowest grade was less than or equal to grade 3 and the highest grade was less than or equal to grade 8. Combined schools contain both elementary and secondary grades (e.g., K to 12 or 1 to 9). See the "Sample Selection" section of this appendix for a detailed description of school types.

Special education classes for children prior to kindergarten – Includes classes that serve only children with Individualized Education Programs (IEPs).

Survey Sponsorship and Acknowledgments

The survey was performed under contract with Westat. Bernie Greene was the NCES Project Officer. The data were requested by the National Institute on Early Childhood Development and Education, Office of Education Research and Improvement, U.S. Department of Education.

This report was reviewed by the following individuals:

Outside NCES

• Margaret Burchinal, University of North Carolina, Chapel Hill
   
• W. Steven Barnett, Rutgers University
   
• Stephanie Cronen, American Institutes for Research, Education Statistics Services Institute
   
• Lawrence Lanahan, American Institutes for Research, Education Statistics Services Institute
   
• Robert Pianta, University of Virginia

Inside NCES

• Stephen Broughman, Elementary/Secondary and Libraries Studies Division
   
• Shelley Burns, Early Childhood, International, and Crosscutting Studies Division
   
• Elvira Germino-Hausken, Early Childhood, International, and Crosscutting Studies Division
   
• Arnold Goldstein, Assessment Division
   
• William Hussar, Early Childhood, International, and Crosscutting Studies Division
   
• Val Plisko, Associate Commissioner, Early Childhood, International, and Crosscutting Studies Division
   
• John Ralph, Early Childhood, International, and Crosscutting Studies Division
   
• Marilyn Seastrom, Chief Statistician, Statistical Standards Program, Office of the Deputy Commissioner
   
• Bruce Taylor, Statistical Standards Program, Office of the Deputy Commissioner
   

List of Appendix Tables

• Table A - 1: Distribution of regular and special education schools by status of prekindergarten enrollment in the 1998–99 NCES Common Core of Data (CCD) Public Elementary and Secondary School Universe file.
   
• Table A - 2: Number and percentage distribution of regular elementary and special education public schools in the respondent sample, estimated number and percentage distribution of regular elementary and special education public schools in the nation, number and percentage distribution of regular elementary and special education public schools with prekindergarten classes in the respondent sample, and estimated number and percentage distribution of regular elementary and special education public schools with prekindergarten classes in the nation: 2000–2001
   

---

⁴²A statement was added to the questionnaire describing the survey focus as being on "classes that serve children prior to kindergarten, regardless of whether your school refers to those classes as prekindergarten, preschool, or some other name."

⁴³The survey questionnaire was also reviewed and approved by the Office of Management and Budget (OMB). The OMB clearance number was 1850-0733.

Top