Technical/Methodology Report:
1993-94 Schools and Staffing Survey: Sample Design and Estimation
October 1996
(NCES 96-089) Ordering information
Highlights
1.1 General Goals
The (NCES) sponsors the Schools and Staffing Survey (SASS) in order to provide periodic, timely data on public and private elementary and secondary schools in the United States. Data collected include school and teacher characteristics, school operations, programs and policies, teacher demand and supply, and the opinions of teachers and administrators about policies and working conditions. These SASS components are: "Teacher Demand and Shortage Survey", the "School Survey", the "School Administrator Survey", the "Teacher Survey", the "Library Survey", the "Librarian Survey", and the "Student Survey". These surveys are all collected during the same school year. Additionally, the Teacher Follow-up Survey (TFS) is conducted on a subsample of the Teacher Survey sample one year after the Teacher Survey is conducted. The integration of all of these elements produces files that can provide linkage of data between the LEAs (local education agencies), schools, principals, libraries, librarians, teachers, and students. To accomplish this integration:
1. Schools were selected first. Each selected school received a school questionnaire and an administrator questionnaire.
2. A sample of school libraries and librarians was selected from the school sample. Each received a library as well as a librarian questionnaire.
3. A sample of teachers was selected within each selected school. The average teacher sample size per school was approximately five. Each selected teacher received a teacher questionnaire.
4. A subsample of schools for the student sample was selected from the school sample. A subsample of three teachers was selected from the sampled teachers in 3 above, within the student sample schools. A sample of two students from each teacher was selected.
5. For public schools, the LEAs associated with the selected schools received a Teacher Demand and Shortage (TDS) questionnaire. An additional sample of districts not associated with schools was selected and received the TDS questionnaire. The school questionnaire for private schools included TDS questions for the school. (See Figure 1A for an illustration of the 1993-94 SASS sampling process.)
The SASS was first conducted by the Bureau of the Census during the 1987-88 school year and was repeated for the 1990-91 school year. This report documents the sample design and estimation procedures for the third SASS collection. It was conducted during the 1993-1994 school year, and is referred to as 1994 SASS in this document. Some 13,000 schools and administrators, and 67,000 teachers were selected. In addition, 5,500 local education agencies associated with the selected schools and 100 districts not associated with schools were selected in 1994 SASS. Some 7,600 school libraries and librarians, and 6,900 students were also selected.
The SASS is designed to provide the following estimates to meet its analytic goals:
1. national estimates for public and private schools;
2. state estimates for public schools, libraries, and librarians;
3. state/elementary, state/secondary, and national combined public school estimates (see section 5.1.2 for the definition of elementary, secondary and combined schools);
4. detailed association estimates and grade level estimates for private schools;
5. estimates of change from 1991 to 1994 in school level characteristics; Figure 1a.--The 1993-94 SASS Sampling Process
6. national estimates of public and private student demographic information;
7. national estimates for schools with greater than 19.5% American Indian or Alaskan Native enrollment (Sometimes referred to simply as Indian);
8. national estimates for schools, libraries, librarians and students from schools operated by the Bureau of Indian Affairs (BIA);
9. national estimates for public and private school libraries, librarians, and students by school grade level and urbanicity;
10. national estimates for private school libraries, librarians, and students by major affiliation (Catholic, other religious, and nonsectarian);
This report describes the procedures used for the: 1. school and teacher sample stratum allocation, 2. overlapping 1991 and 1994 SASS samples, 3. public school and principal sample design, 4. LEA sample design, 5. private school and principal sample design, 6. teacher sample design (including within school teacher allocation), 7. public and private school library and librarian sample design, 8. public and private student sample design, 9. weighting, 10. imputation, 11. variance estimation techniques, and 12. frame evaluation.
1.2 Response Rates
Below are the unweighted and weighted questionnaire response rates for the SASS components, as well as the weighted overall response rates. The unweighted questionnaire response rates are defined as the number of in-scope (eligible for interview) responding questionnaires divided by the number of in-scope sample cases. The weighted questionnaire response rates are defined the same way, using the basic weighted (inverse of the probability of selection) instead of unweighted numbers. The weighted overall response rates are defined as the weighted questionnaire response rates times the rate at which the sample schools cooperated with the sampling at each stage of the sampling. The overall response rate for a particular item (overall response rate times item response rate) may be lower than the respective response rates because the individual item nonresponse rates are not included in Table 1.
Table 1.--Weighted and Unweighted Questionnaire Response Rates and Weighted Overall Response Rates
Unweighted Weighted
Weighted
Questionnaire Questionnaire
Overall
Survey Type Response Rate Response Rate/1
Response Rate/2
-------------------------------------------------------------------------------
---------
Teacher Demand and Shortage (LEA) 93.1 93.9
93.9
Public School Administrator 96.6 96.6
96.6
Private School Administrator 90.3 87.6
87.6
Indian School Administrator 98.7 98.7
98.7
Public School 92.0 92.3
92.3
Private School 84.1 83.2
83.2
Indian School 99.3 99.3
99.3
Public Teacher 88.9/3 88.2/3
83.8
Private Teacher 80.6/4 80.2/4
73.0
Indian Teacher 87.1 86.6
86.6
Public School Library 91.1 90.1
90.1
Public School Librarian 93.5 92.3
92.3
Private School Library 77.7 70.7
70.7
Private School Librarian 83.9 76.5
76.5
Indian Library 89.4 89.4
89.4
Indian Librarian 88.3 88.3
88.3
Public School Student 90.2/5 91.3/5
80.3
Private School Student 87.6/6 88.1/6
69.6
Indian School Student 93.7/7 92.5/7
87.0
-------------------------------------------------------------------------------
---------
1/Weighted using the inverse of the probability of selection.
2/Weighted Questionnaire Response Rate times the rate of cooperation with the sampling of the sample schools at each stage of the selection.
3/These rates do not include the 5 percent of public schools that did not provide teacher lists.
4/These rates do not include the 9 percent of private schools that did not provide teacher lists.
5/These rates do not include the 12 percent of public schools that did not participate in student sampling.
6/These rates do not include the 21 percent of private schools that did not participate in student sampling.
7/These rates do not include the 6 percent of Indian schools that did not participate in student sampling.
Source: 1993-94 Schools and Staffing Survey - all components.
Table 2.-- Unweighted Overlap/Nonoverlap Questionnaire Response Rates
Survey Type Overlap Response Rate Nonoverlap Response Rate ----------------------------------------------------------------- Public School 91.8 92.1 Private School 87.9 82.8 -----------------------------------------------------------------
Source: 1993 - 94 Schools and Staffing Survey - School Components.
A future report will examine survey response rates and possible bias in more detail for the 1993-94 SASS. An examination of non-response on the 1990-91 SASS can be found in Scheuren et al. (1996) and Monaco et al. (1996).
Table 2 provides the 1994 unweighted response rates for schools being asked to respond to SASS in: 1) both 1991 and 1994 (overlap units), and 2) 1994 only (nonoverlap). See section 4 for more information concerning the selection of overlap schools.
1.3 Changes in SASS Design from 1988 to 1994
Various changes were made to the SASS design between survey years 1988 to 1991 and 1991 to 1994 to improve SASS estimates. The 1991 to 1994 changes are discussed below, along with the implications of the design changes over the years.
1.3.1 Changes in SASS Design from 1991 to 1994
Below is a summary of the changes made in the 1994 sample:
1. To improve the precision of the 1994 private sector estimates, the frame resulting from the 1991-92 Private School Survey (PSS) updated with affiliation list matching results (See Section 5.3.2) was used as the sampling frame for the 1993-94 SASS private schools. See appendix 1 for more information about PSS. The 1994 private school stratum definitions were based on the 1991-92 PSS school reports of association membership and affiliation.
2. Private school weights were adjusted so that 1993-94 SASS school totals would agree with 1993-94 PSS school totals. See section 9.1 for an explanation for why this was done.
3. For the private sector, the sample was reallocated to publish estimates for one additional association - for a total of 19 associations.
4. In 1994, a library/librarian survey was initiated, as well as a student survey.
5. The cutoff for the Native American schools was changed to an enrollment greater than 19.5% instead of 25%.
6. The schools in the BIA stratum were selected with certainty.
7. CATI facilities were used extensively for the nonresponse follow-up of the teacher survey, librarian survey, library survey, public school survey, and administrator survey.
8. Teacher lists from sample schools were keyed, allowing for better control over sample sizes by stratum and improving the effectiveness of the sort.
9. Administrators who teach were eligible for the teacher sample, in addition to receiving an administrator questionnaire.
10. Collapsing criteria were altered slightly for the LEA weighting. See Section 9.4 for more details.
1.3.2 Concerns about SASS change estimates from 1988 to 1994.
Care must be taken when estimating change from 1988 to 1994 in a SASS data element, because some of the change may be due to change in sample design, as opposed to change in the education system (for example a 30% increase in the number of schools in Nebraska). Below are sample design changes that might affect the measurement of change over time.
1.3.2.1 Changing the sampling frame from QED to CCD
Beginning with the 1990-91 SASS, the sampling frame for public schools was changed. A change in the sampling frame is of some concern because the definition of a school is different between the two frames. In the 1987-88 SASS, a school was defined as a physical location based in information included in the Quality Education Data (QED) file. In the 1990-91 SASS, a school was defined as an administrative unit with a principal based on information included in the Common Core of Data (CCD) file. In states which have multiple administrative units in a single physical location, the estimated change in the number of schools could increase. This increase is at least partially caused from the definition difference.
It is possible to collapse the 1993-94 SASS school sample to reflect the QED definition of a school as defined in the 1987-88 SASS, thereby eliminating this concern. However, resulting estimates will no longer be consistent with CCD estimates.
To the extent that the coverage between CCD and QED are different, then part of the change in school related estimates can be attributable to this coverage difference.
1.3.2.2 Adjusting the estimated number of teachers from the teacher file to the estimated number of teachers from the school file
This was done, beginning with 1990-91 SASS, to make estimates in the files more consistent. Since this was not done in the 1987-88 survey, some observed distributional differences between the 1987-88 and 1994 teacher files may be partially attributable to this adjustment. In the public 1987-88 SASS files, the teacher counts on the teacher file are smaller than the counts on the school file. In the 1993-94 SASS files, the teacher file counts are increased to equalize the estimates. This increase does not reflect a change in the educational system, but a bias correction differentially applied between the files.
1.3.2.3 Imputing for missing data on the administrator and teacher files
All data files are adjusted for complete refusals. However, for the 1988 administrator and teacher files, missing data elements within responding units are not imputed. Hence, estimates of totals implicitly use a value of zero for all missing data elements (i.e., 1988 totals are underestimates whenever there are missing data). Beginning with 1991, and again for 1994, estimates of totals use imputed values for missing data elements. Therefore, some of the measured change between the 1988 and 1994 totals is due to imputing one year, but not the other. This part of the change is not due to a change in the educational system.
Change estimates for ratios and averages are also affected by imputations in one year, but not the other. However, the magnitude and direction of the bias is unknown and dependent on the variable of interest. This part of the change is not due to a change in the educational system.
1.3.2.4 Questionnaire and conceptual differences
Care must be observed in the interpretation of change estimates between 1988 and 1994, since specific questions are not always worded the same from the first SASS survey to subsequent surveys. Both major and minor changes in wording of specific items occur; the ordering of items may be different and concepts can be different.
As an example, in both the 1987-88 and 1990-91 SASS, the question, "Which best describes the community in which the school is located?" was asked of the principal (for the school survey) and the respondent to the school survey. The SASS re-interview program in both 1988 and 1991 determined the responses to this item were highly subjective and exhibited moderate response variance. As a result of this finding, the 1991 and 1994 SASS micro-data files contain an "urbanicity" code (Locale) developed by Johnson (1989). This code is believed to be a more accurate description of the community than the self-reports on SASS. This methodology assigns "type of locale codes" based on the school mailing address matched to Bureau of the Census data files containing population density data, Standard Metropolitan Statistical Area (SMSA) codes, and a Census code defining urban and rural areas.
This rigorously defined locale code on the 1990-91 and 1993-94 SASS files may be different from the self-report of community type.
Download/view the full report in a PDF file. (file size: 1,329K)
For more information about the contents of this report, contact Steve Kaufman at (202) 202-502-7371 Steve.Kaufman@ed.gov.