Appendix A—Survey Methodology and Data Reliability

Sample Selection

The sampling frame for the FRSS School Survey on Racial and Ethnic Classifications was constructed from the 1992-93 NCES Common Core of Data (CCD) public school universe file and included over 79,000 public elementary and secondary schools. Excluded from the frame were special education, vocational, and alternative/other schools, schools outside the 50 states and the District of Columbia, and schools whose highest grade was less than first grade.

A stratified sample of 1,000 schools-500 elementary and 500 secondary-was selected for the survey. The sample was stratified by geographic region (Northeast, Southeast, Central, and West), metropolitan status (city, urban fringe, town, and rural), percent minority enrollment (less than 5, 5-19, 20-49, and 50 or greater), and school size (less than 300, 300-499, 500-999, and 1,000 or more). The sample sizes were then allocated to the primary strata in rough proportion to the aggregate square root of the enrollment of schools in the stratum. The use of the square root of enrollment to determine the sample allocation was expected to be reasonably efficient for estimating both schoollevel characteristics (e.g., percentage of schools that use additional racial and ethnic classifications) and quantitative measures correlated with enrollment (e.g., the number of students whom they feel are not accurately described by the five standard federal categories). Further, the sample sizes were large enough to permit analyses of the questionnaire (along one dimension) by the four regions, four urbanicity classes, four levels of minority enrollment, and four enrollment size classes (table 7).

Respondents and Response Rates

In early May 1995, questionnaires (see appendix C) were mailed to 500 public elementary school principals and 500 secondary school principals. The principal was asked either to complete the questionnaire or to have it completed by the person in his or her school who was most knowledgeable about the collection, recording, and reporting of information regarding the race and ethnicity of the school's student body. Principals completed 72 percent of the questionnaires, other administrators completed 24 percent, district representatives completed 4 percent of the questionnaires, and teachers completed less than 1 percent. Eight schools were found to be out of scope (no longer at the same location or serving the same population), leaving 992 eligible schools in the sample. Telephone followup of nonrespondents was initiated in mid- May; data collection was completed by June 1995. Sixty percent of the questionnaires were returned by mail, 25 percent were completed by phone, and 16 percent were submitted by fax. A total of 926 schools completed the survey.

Thus, the final response rate was 93 percent. The weighted response rate was also 93 percent. Item nonresponse ranged from 0.0 to 0.9 percent.

Sampling and Nonsampling Errors

The response data were weighted to produce national estimates. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in collection of the data. These errors can sometimes bias the data. Nonsampling errors may include such problems as the differences in the respondents' interpretation of the meaning of the questions; memory effects; misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

To minimize the potential for nonsampling errors, the questionnaire was pretested with public school principals. During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were extensively reviewed by staff at the National Center for Education Statistics. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Imputations for item nonresponse were not implemented, as item nonresponse rates were very low (less than 1 percent). Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of public schools reporting that they only use the five standard federal categories for collecting race and ethnicity information is 73 percent, and the estimated standard error is 1.4 percent. The 95 percent confidence interval for the statistic extends from [73 - (1.4 x 1.96) to 73 + (1.4 x 1.96)], or from 70.2 to 75.4.

Estimates of standard errors were computed using a technique known as jackknife replication, which accounts for the complexities of the sample design. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic (see Wolter 1985, Chapter 4). To construct the replications, 50 stratified subsamples of the full sample were created and then dropped one at a time to define 50 jackknife replicates (see Wolter 1985, page 183). A proprietary computer program (WESVAR), available at Westat, Inc., was used to calculate the estimates of standard errors. The software runs under IBM/OS and VAX/VMX systems.

Background Information

The survey was performed under contract with Westat, Inc., using the NCES Fast Response Survey System (FRSS). Westat's Project Director was Elizabeth Farris, and the Survey Manager was Nancy Carey. Judi Carpenter was the NCES Project Officer. The data were requested by Edith McArthur of NCES and Sharon Tuchman of OCR in the Department of Education.

This report was reviewed by the following individuals:

Outside NCES

• Suzann Evinger and Katherine Wallman, Office of Management and Budget
• Nampeo McKenney and Claudette Bennett, Bureau of the Census
• Lavan Dukes, Florida Department of Education

Inside NCES

• Kathryn Chandler, Bill Freund, and Lee Hoffman, Survey and Cooperative Systems Group
• Mary Frase, Data Development and Longitudinal Surveys Group

For more information about the Fast Response Survey System or the School Survey of Racial and Ethnic Classifications, contact Judi Carpenter, Elementary/Secondary Education Statistics Division, Office of Educational Research and Improvement, National Center for Education Statistics.

References and Additional Readings

Department of Commerce. Bureau of the Census. "1996 Race and Ethnic Targeted Test" (RAETT) and its Content Reinterview, Also Identified as the 1996 Census Survey. Federal Registrar (60FR62010-15).

Evinger, S. 1995. "How Shall We Measure Our Nation's Diversity?" Chance, Winter, 7-14.

Executive Office of the President. Office of Management and Budget. "Standards for the Classification of Federal Data on Race and Ethnicity," Federal Register (59FR298 31-35 and 60FR44674-93).

Harrison, R., and Bennett, C. 1995. "Racial and Ethnic Diversity." Chapter 4 in State of the Union, America in the 1990s, Vol. 2, Social Trends. 1990 Monograph Series. Ed. R. Farley, Russell Sage Foundation, New York.

Hodgkinson, H. 1995. "What Should We Call People?" Phi Delta Kappan, October, 173-179.

The WESVAR Procedures. 1989. Rockville, MD: Westat, Inc.

U.S. Department of Labor, Bureau of Labor Statistics. 1995. "A CPS Supplement for Testing Methods of Collecting Racial and Ethnic Information: May 1995."

Wolter, K. 1985. Introduction to Variance Estimation. Springer-Verlag.

Directive No. 15 Categories and Definitions

Terms Defined on the Survey Questionnaire:

American Indian or Alaskan Native - A person having origins in any of the original peoples of North America, and who maintains cultural identification through tribal affiliations or community recognition.

Asian or Pacific Islander - A person having origins in any of the original peoples of the Far East, Southeast Asia, the Indian subcontinent, or the Pacific Islands. This area includes, for example, China, India, Japan, Korea, the Philippine Islands, and Samoa.

Black, not of Hispanic origin - A person having origins in any of the black racial groups of Africa.

Hispanic - A person of Mexican, Puerto Rican, Cuban, Central or South American or other Spanish culture or origin, regardless of race.

White, not of Hispanic origin - A person having origins in any of the original peoples of Europe, North Africa, or the Middle East.

Sample Universe and Classification Variables

The following classification variables come from NCES's Common Core of Data (CCD).

Metropolitan Status

Urban - a central city of a Metropolitan Statistical Area (MSA).

Urban fringe - a place within an MSA of a central city, but not primarily its central city.

Town - a place not within an MSA, but with a population greater than or equal to 2,500, and defined as urban by the U.S. Bureau of the Census.

Rural - a place with a population less than 2,500 and defined as rural by the U.S. Bureau of the Census.

Geographic Region

Northeast - Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.

Southeast - Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.

Central - Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin. West - Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming.

Top