Appendix A Technical Notes

The Fast Response Survey System (FRSS) was established in 1975 by the National Center for Education Statistics (NCES), U.S. Department of Education. FRSS is designed to collect small amounts of issue-oriented data with minimal burden on respondents and with a quick turnaround from data collection to reporting.

Sample Selection

The sample of elementary and secondary schools for the "Internet Access in U.S. Public Schools, Fall 2003" was selected from the 2001–02 NCES Common Core of Data (CCD) Public Elementary/Secondary School Universe File, the most up-to-date file available at the time the sample was drawn. Over 95,000 schools are contained in the 2001–02 CCD Public Elementary/Secondary School Universe File. For this survey, regular elementary and secondary/combined schools were selected. Special education, vocational education, and alternative schools were excluded from the sampling frame, along with schools with a highest grade below first grade and those outside the 50 states and the District of Columbia. With these exclusions, the final sampling frame consisted of about 84,000 schools, of which about 63,000 were classified as elementary schools and about 21,000 as secondary/combined schools.¹

A sample of 1,207 schools was selected from the public school frame. To select the sample, the frame of schools was stratified by instructional level (elementary, secondary/combined schools), enrollment size (less than 300 students, 300 to 499, 500 to 999, 1,000 to 1,499, 1,500 or more), and percentage of students eligible for free or reduced-price lunch (less than 35 percent, 35 to 49 percent, 50 to 74 percent, 75 percent or more). Schools in the highest poverty category (schools with 75 percent or more students eligible for free or reduced-price lunch) were oversampled to permit analyses for that category.

Respondents and Response Rates

The three-page survey instrument was designed by Westat and NCES to address all of the issues examined in the 2002 survey on Internet access. These issues included access to the Internet in instructional rooms, the types of Internet connections used, student access to the Internet outside of regular school hours, laptop loans, hand-held computers for students and teachers, school websites, teacher professional development on how to integrate the use of the Internet into the curriculum, and technologies and procedures used to prevent student access to inappropriate material on the Internet.

Questionnaires and cover letters were mailed to the principals of the 1,207 sampled schools in early October 2003. The letter introduced the study and requested that the questionnaire be completed by the technology coordinator or person most knowledgeable about Internet access at the school. Respondents were offered the option of completing the survey via the Web or by mail. Telephone followup for survey nonresponse and data clarification was initiated in October 2003, and data collection was completed in February 2004. Fourteen schools were outside the scope of the survey, and 1,081 schools completed the survey. Thus, the final response rate was 91 percent (1,081 of 1,193 eligible schools). The weighted response rate was 92 percent.

Imputation for Nonresponse

The weighted item nonresponse for questionnaire items was less than 1 percent. The nonresponse rate for a particular item was calculated using the number of responses as the numerator and the estimated number of eligible cases that should have responded to the item as the denominator. Although item nonresponse for key items was very low, missing data were imputed for the 20 items listed in table Table A-1. No imputation was done for school characteristic variables (e.g., percent minority enrollment) that were created from CCD data. The missing items included both numerical data such as counts of instructional rooms and computers, as well as categorical data such as the provision of handheld computers to students and teachers. The missing data were imputed using a "hot-deck" approach to obtain a "donor" school from which the imputed values were derived. Under the hot-deck approach, a donor school that matched selected characteristics of the school with missing data was identified. The matching characteristics included level, enrollment size class, type of locale, and total number of computers in the school. Once a donor was found, it was used to derive the imputed values for the school with missing data. For categorical items, the imputed value was simply the corresponding value from the donor school. For numerical items, an appropriate ratio (e.g., the proportion of instructional rooms with Internet access) was calculated for the donor school, and this ratio was applied to available data (e.g., reported number of instructional rooms) for the recipient school to obtain the corresponding imputed value. All missing items for a given school were imputed from the same donor.

Sampling and Nonsampling Errors

The survey responses were weighted to produce national estimates (Table A-2). The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are based on the sample selected and, consequently, are subject to sampling variability. The standard error is the measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of instructional rooms with Internet access in 2003 is 93 percent, and the estimated standard error is 0.5 percent. The 95 percent confidence interval for the statistic extends from 93 - (0.5 times 1.96) to 93 + (0.5 times 1.96), or from 92 to 94 percent. The coefficient of variation ("c.v.," also referred to as the "relative standard error") expresses the standard error as a percentage of the quantity being estimated. The c.v. of an estimate (y) is defined as c.v. = (s.e./y) x 100. Throughout this report, for any coefficient of variation higher than 50 percent, the data are flagged with the note that they should be interpreted with caution, as the value of the estimate may be unstable.

Because the data from this survey were collected using a complex sampling design, the sampling errors of the estimates from this survey (e.g., estimates of proportions) are typically larger than would be expected based on a simple random sample. Not taking the complex sample design into account can lead to an underestimation of the standard errors associated with such estimates. To generate accurate standard errors for the estimates in this report, standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. To construct the replications, 50 stratified subsamples of the full sample were created and then dropped one at a time to define 50 jackknife replicates. A computer program (WesVar) was used to calculate the estimates of standard errors. WesVar is a stand-alone Windows application that computes sampling errors from complex samples for a wide variety of statistics (totals, percents, ratios, log-odds ratios, general functions of estimates in tables, linear regression parameters, and logistic regression parameters).

The test statistics used in the analysis were calculated using the jackknife variances and thus appropriately reflect the complex nature of the sample design. In particular, Bonferroni adjustments were made to control for multiple comparisons where appropriate. For example, for an "experiment-wise" comparison involving g pairwise comparisons, each difference was tested at the 0.05/g significance level to control for the fact that g differences were simultaneously tested. The Bonferroni adjustment was also used for previous FRSS Internet reports. The Bonferroni adjustment is appropriate to test for statistical significance when the analyses are mainly exploratory (as in this report) because it results in a more conservative critical value for judging statistical significance. This means that comparisons that would have been significant with a critical value of 1.96 may not be significant with the more conservative A-8 critical value. For example, the critical value for comparisons between any two of the four categories of poverty concentration is 2.64 rather than 1.96.

When comparing percentage or ratio estimates across a family of three or more ordered categories (e.g., categories defined by percent minority enrollment), regression analyses were used to test for trends rather than a series of paired comparisons. For proportions, the analyses involved fitting models in WesVar with the ordered categories as the independent variable and the (dichotomous) outcome of interest (e.g., whether or not the school made computers with Internet access available before school) as the dependent variable. For testing the overall significance, an analysis of variance (ANOVA) model was fitted by treating the categories of the independent variables as nominal categories. For the trend test, a simple linear regression model was used with the categories of the independent variable as an ordinal quantitative variable. In both cases, tests of significance were performed using an adjusted Wald F-test. The test is applicable to data collected through complex sample surveys and is analogous to Ftests in standard regression analysis. For estimated ratios, similar tests of overall significance and linear trends were performed using procedures analogous to those described by Skinner, Holt, and Smith.² A test was considered significant if the p-value associated with the statistic was less than 0.05.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in collection of the data. These errors can sometimes bias the data. Nonsampling errors may include such problems as the difference in the respondents' interpretation of the meaning of the question; memory effects; misrecording of responses; incorrect editing, coding, or data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used. To minimize the potential for nonsampling errors, the questionnaire on Internet access in public schools was pretested in 1994, and again each time it was substantially modified. The questionnaire was last pretested for the fall 2001 survey, since a few new topics were introduced in the survey. The pretesting was done with public school technology coordinators and other knowledgeable respondents like those who would complete the survey. During the design of the survey, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were intensively reviewed by NCES.

Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone to resolve problems. Data were keyed with 100 percent verification.

Definitions of Terms Used in the Questionnaire

Types of Internet connections

T3/DS3-Dedicated digital transmission of data and voice at the speed of 45 MB per second; composed of 672 channels.

Fractional T3-One or more channels of a T3/DS3 line. Used for data and voice transmission at the speed of less than 45 MB per second.

T1/DS1-Dedicated digital transmission of data and voice at the speed of 1.5 MB per second; composed of 24 channels.

Fractional T1-One or more channels of a T1/DS1 line. Used for data and voice transmission at the speed of less than 1.5 MB per second.

Cable modem-Dedicated transmission of data through cable TV wires at a speed of up to 2 MB per second.

DSL (Digital Subscriber Line)-Refers collectively to ADSL, SDSL, HDSL, and SDSL. DSLs have a dedicated digital transmission speed of up to 32 MB per second.

ISDN (Integrated Services Digital Network)-Sends voice and data over digital telephone lines or normal telephone wires at the speed of up to 128 KB per second.

56 KB-Dedicated digital transmission of data at the speed of 56 KB per second.

Dial-up connection-Data transmission through a normal telephone line upon command, at the maximum speed of 56 KB per second (for example, AOL or Earthlink).

Types of technologies to prevent student access to inappropriate material on the Internet

Blocking software-Uses a list of websites that are considered inappropriate and prevents access to those sites.

Filtering software-Blocks access to sites containing keywords, alone or in context with other keywords.

Monitoring software-Records e-mails, instant messages, chats, and the websites visited.

Intranet-Controlled computer network similar to the Internet, but accessible only to those who have permission to use it. Intranet system managers can limit user access to Internet material.

Definitions of Analysis Variables

Instructional level-Schools were classified according to their grade span in the 2001–02 Common Core of Data (CCD) Public Elementary/Secondary School Universe File. Data for combined schools are included in the totals and in analyses by other school characteristics, but are not shown separately. Thus, data are reported for the following categories:

Elementary school-Had grade 6 or lower and no grade higher than grade 8.

Secondary school-Had no grade lower than grade 7 and had grade 7 or higher.

School size-This variable indicates the total enrollment of students based on data from the 2001–02 CCD Public Elementary/Secondary School Universe File. For sampling purposes, schools were grouped into five enrollment size classes-less than 300 students, 300 to 499, 500 to 999, 1,000 to 1,499, 1,500 or more. Use of the more detailed size categories ensures greater diversity of schools in the sample with respect to size, and permits a more nearly optimal allocation of the sample for estimating school-level characteristics that are correlated with enrollment. Because of the relatively small sample size and large standard errors associated with small cell sizes, the following three combined categories were used for analysis purposes:

Less than 300 students
300 to 999 students
1,000 or more students

Locale-This variable indicates the type of community in which the school is located, as defined in the 2001–02 CCD Public Elementary/Secondary School Universe File (which uses definitions based on U.S. Census Bureau classifications). The variable was based on the eight-category locale variable from CCD and collapsed into the following four categories for this report.

City-A central city of a Consolidated Metropolitan Statistical Area (CMSA) or Metropolitan Statistical Area (MSA).

Urban fringe-Any incorporated place, Census-designated place, or non-place territory within a CMSA or MSA of a large or mid-size city and defined as urban by the Census Bureau.

Town-An incorporated place or Census-designated place with a population greater than or equal to 2,500 and located outside a CMSA or MSA.

Rural-Any incorporated place, Census-designated place, or non-place territory designated as rural by the Census Bureau.

Percent minority enrollment-This variable indicates the percent of students enrolled in the school whose race or ethnicity is classified as one of the following: American Indian or Alaskan Native; Asian or Pacific Islander; Black, non-Hispanic; or Hispanic, based on data in the 2001–02 CCD Public Elementary/Secondary School Universe File. The categories are:

Less than 6 percent
6 to 20 percent
21 to 49 percent
50 percent or more

Percent of students eligible for free or reduced-price lunch-This variable was based on responses to question 32 on the survey questionnaire; if it was missing from the questionnaire (1.7 percent of all cases), it was obtained from the 2001–02 CCD Public Elementary/Secondary School Universe File. This item served as a measurement of the concentration of poverty at the school. The categories are:

Less than 35 percent
35 to 49 percent
50 to 74 percent
75 percent or more

Geographic region-This variable was obtained from the 2001–02 CCD Public Elementary/Secondary School Universe File. It classifies schools into one of the following four regions used by the Bureau of Economic Analysis of the U.S. Department of Commerce, the National Assessment of Educational Progress, and the National Education Association.

Northeast-Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.

Southeast-Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia.

Central-Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin.

West-Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, Washington, and Wyoming.

It is important to note that many of the school characteristics used for independent analysis may also be related to each other. For example, school size and locale are related, with city schools typically being larger than rural schools. Similarly, poverty concentration and minority enrollment are related, with schools with a higher minority enrollment also more likely to have a higher concentration of poverty. Other relationships between analysis variables may exist. However, this E.D. TAB report focuses on bivariate relationships between the analysis variables and questionnaire variables rather than more complex analyses.

For more information about the survey, contact Bernard Greene, Early Childhood, International, and Crosscutting Studies Division, National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, 1990 K Street NW, Washington, DC 20006; e-mail: Bernard.Greene@ed.gov; telephone: (202) 502-7348.

---

¹During data collection, a number of sampled schools were found to be outside the scope of the survey, usually because they were closed or merged. This reduced the number of schools in the sampling frame to an estimated 82,036.

²Skinner, C.J., Holt, D., and Smith, T.M.F. (1989). Analysis of Complex Surveys. Chichester, England: John Wiley & Sons.

Top