Survey Methodology and Data Reliability Sample Selection

The sample of public schools for the FRSS Principal/School Disciplinarian Survey on School Violence was selected from the 1993-94 NCES Common Core of Data (CCD) Public School Universe File. Over 84,000 public schools are contained in the CCD Universe File, of which almost 79,000- 49,000 regular elementary schools, 14,000 regular middle schools, and 15,801 regular secondary/combined schools in the 50 states and the District of Columbia-meet the eligibility criteria for this study. Excluded from the sampling frame were special education, vocational, and alternative/other schools, schools in the territories, and schools with a high grade lower than one or ungraded.

The sample was stratified by instructional level (elementary, middle, and secondary/combined), locale (city, urban fringe, town, rural), and school size (less than 300, 300 to 999, and 1,000 or more; Table A). Within the primary strata, schools were also sorted by geographic region (Northeast, Southeast, Central, West) and by percent minority enrollment (less than 5 percent or missing, 5 to 19 percent, 20 to 49 percent, and 50 percent or more). The sample sizes were then allocated to the primary strata in rough proportion to the aggregate square root of the size of enrollment of schools in the stratum.

The use of the square root of the size of enrollment to determine the sample allocation is considered efficient for estimating school-level characteristics (e.g., number or percent of schools that reported an incident of robbery occurred at their school). The sample sizes were large enough to permit limited analysis of the questionnaire (along one dimension) by the four regions, four locales, three enrollment size categories, five levels of poverty concentration, and four categories of minority enrollment, but not the independent effects of each characteristic.

Respondent and Response Rates

In April 1997 questionnaires were mailed to 1,415 school principals. The principal was asked either to complete the questionnaire or to have it completed by the school disciplinarian who was most knowledgeable about discipline issues at the school. Telephone followup of nonrespondents was initiated in late April, and data collection was completed in July 1997.

Eleven schools were found to be out of the scope of the study (no longer in existence), and a total of 1,234 schools completed the survey. Thus, the final raw response rate was 88 percent (1,234 schools divided by the 1,404 eligible schools in the sample). The weighted overall response rate was 89 percent. Weighted item nonresponse rates ranged from 0 percent to 0.9 percent. Because the item nonresponse was so low, imputation for item nonresponse was not implemented.

Comparisons with principals' perceptions about school discipline in 1991 used the results the 1991 FRSS Principal Survey on Safe, Disciplined, and Drug-Free Schools. That survey was mailed to a sample of 890 public schools in April 1991. Six of the schools were found to be closed, leaving 884 schools in the sample. Telephone followup commenced in mid-May; data collection was completed by the end of June, 1991. A response rate of 94 percent was achieved (830 responding principals divided by 884 principals in the sample) for the 1991 study and item nonresponse ranged from 0.0 percent to 3.1 percent.

Because of small sampling differences between the 1991 and 1996-97 surveys, it was preferable not to simply make comparisons with data provided in the 1991 survey report; consequently, new analyses were run on the 1991 data file. The 1991 survey design had included regular, vocational education, and alternative schools in the sample, while the 1996-97 survey included only regular schools and excluded vocational and alternative schools from the sampling frame. Thus, additional analyses were done dropping vocational and alternative schools from the 1991 data set so that the samples would be comparable. Thirteen vocational and alternative schools were dropped from the analyses, and all 1991 data were recalculated on a sample of 817 regular public schools.

Sampling and Nonsampling Errors

The responses were weighted to produce national estimates. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings of this survey are estimates based upon the sample selected and, as a result, are subject to sampling variability.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in collection of the data. These errors can sometimes bias data. Nonsampling errors may include such problems as the differences in the respondents' interpretation of the meaning of the questions; memory effects; misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used. To minimize the potential for nonsampling errors, the questionnaire was pretested with public school principals like those who completed the survey.

During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were extensively reviewed by the National Center for Education Statistics. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of public schools reporting any incidence of crime is 57 percent, and the estimated standard error is 2.1 percent. The 95 percent confidence interval for the statistic extends from [57-(2.1 times 1.96)] to [57+ (2.1 times 1.96)] or from 52.8 to 61.1 percent.

Estimates of standard errors for this report were computed using a technique known as a jackknife replication method. Standard errors for all of the estimates are presented in the tables. All specific statements of comparison made in this report have been tested for statistical significance through tests adjusted for multiple comparisons using the Bonferroni adjustment, and they are significant at the 95 percent confidence level or better.

Top