- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Technical Notes
VI. Imputation
For questionnaire items that should have been answered but were not, values were imputed by using data from (1) other items on the questionnaire, (2) the 1993-94 SASS Teacher Survey record for the same respondent, and (3) data from the record for a respondent with similar characteristics (commonly known as the nearest neighbor "hotdeck" method for imputing for item nonresponse/17).
For some incomplete items, the entry from another part of the questionnaire, the SASS Teacher Survey record, or the data record for a similar case was directly imputed to complete the item; for others, the entry was used as part of an adjustment factor with other data on the incomplete record.
The procedures described above were carried out by computer processing. However, for a few items there were cases where entries were clerically imputed. The data record, SASS teacher file record, and in some cases, the questionnaire were reviewed and an entry consistent with the information from those sources was imputed. This procedure was used when (1) there was not suitable record to use as a donor, (2) the computer method produced an entry that was outside the acceptable range for the item, or (3) there were very few cases where an item was unanswered (usually less than 10).
Values were imputed to items with missing data within records classified as interviews (ISR=1). Noninterview adjustment factors were used during the weighting process to compensate for data missing because the sample person was a noninterview (ISR=2).
Entries imputed to TFS records are identified by flags that denote the stage or type of imputation: 1 = ratio adjustment of original entry; 2 = entry was imputed by using other data on the record or from the SASS teacher file; 3 = entry was imputed by using data from the record for a similar sample person (donor); 4 = clerical imputation; 0 = not imputed.
The variable names for these flags are F_ (variable name), where variable name is the variable name for the data entry, e.g., F_TFS012 is the imputation flag for variable TFS012 (item 6 of the TFS-2).
VII. Weighting
The SASS teacher basic weight is the inverse of the probability of selection of the teacher. Teacher basic weights were adjusted to account for schools that refused to provide lists of teachers (school nonresponse adjustment factor), and for teachers who were selected for the survey but did not provide questionnaire data (teacher noninterview factor). In addition, the school sampling adjustment factor and the first-stage ratio adjustment factor were also applied to produce the final weight.
School sampling adjustment factor was applied to certain schools to account for duplicate records, merged schools, or any other circumstance that would affect the school's true probability of selection.
School nonresponse adjustment factor was calculated to compensate for schools that refused to provide lists of their teachers.
First stage ratio adjustment factor adjusted the sample weighted count of all cases (interviewed, noninterview, and out-of-scope) to known frame totals. For public schools, the frame totals such as grade level by urbanicity by state came from the 1991-92 CCD. For private schools on the list frame, the updated private school list frame universe was the source of totals such as grade level by association membership.
The final TFS sample weight equals:
TFS basic weight x SASS weighting adjustment factor x TFS noninterview adjustment x TFS ratio adjustment where:
TFS basic weight is the inverse of the probability of selecting a teacher for TFS. This weight is the product of the intermediate teacher weight from SASS (described in previous section) and TFS subsampling adjustment factor. The TFS subsampling adjustment factor is an adjustment that accounts for the subsampling of teachers from SASS sample teachers.
SASS weighting adjustment factor is used to adjust for the fact that preliminary SASS final weights were used in computing the TFS basic weight. The weighting adjustment factor adjusts for any changes that may have occurred between the preliminary and final weighting calculations.
TFS noninterview adjustment is the factor used to adjust for teachers who participated in SASS but did not participate in the 1994-95 TFS.
TFS ratio adjustment is the factor used to adjust the TFS sample totals to known SASS sample totals. This adjustment ensures that the weighted number of
(interviews, noninterviews, and out-of-scopes) will equal the weighted number of SASS teachers from 1993-94.
VIII. Variance Estimation
The previous SASS surveys (1987-88 and 1991-92) used the variance procedure known as balanced half sample replication (BHR). A fundamental problem with BHR is that it assumes sampling is done with replacement. Hence, BHR cannot reflect the increase in precision due to sampling a large proportion of a finite population. For most surveys, where the sampling rates are small, the increase in precision will be small and can be safely ignored. However, in SASS the public surveys (school, principal, teacher, library, and librarian) are designed for reliable state estimates. This necessarily implies large sampling rates, which can lead to very large variance overestimates with BHR. Likewise, some of the private surveys (school, principal, and teacher) are designed to produce detailed private association estimates, which also imply large sampling rates, and variance overestimation with BHR.
To overcome this problem, a bootstrap variance estimator was implemented for the 1993-94 SASS. The bootstrap variance reflects the increase in precision due to large sampling rates.
The idea behind bootstrap variance estimation/18 is to use the distribution of the sample weights to generate a bootstrap frame. Bootstrap samples can be selected from the bootstrap frame, replicate weights computed, and variances estimated with standard BHR software. The bootstrap replicate basic weights (inverse of the probability of selection) were subsequently reweighted by processing each set of replicate basic weights through the full-sample weighting procedure.
Further analysis of the bootstrap replicate basic weights revealed that approximately 6% of SASS school replicate weights fell outside a 95% confidence interval. This is only slightly higher than the expected 5% and indicates the bootstrap replicate weights are close to normally distributed.
Public schools. The SASS public school data files contain a set of 48 bootstrap weights, which can be used with any BHR software package. If the package requires specifying a variance methodology, BHR can be specified. At this point, variance computation is similar to the previous SASS and TFS rounds. The difference is in the use of bootstrap methods to produce the replicate weights.
Public school principal replicate weights are the same as the school replicate weights.
Private schools. For private schools, the list frame used the bootstrap methodology as described above. For the area frame, the PSU sampling rates were very small, negating the advantage of using bootstrap.
BHR methodology was employed in the area frame as it has been for all previous SASS. Half-samples are defined by pairing sample PSUs within each sampling stratum, forming variance strata. The final product is a set of 48 replicate weights. After the variance strata were assigned, an orthogonal matrix was used to form the 48 balanced half-sample replicates. Thus, the same methodology can be applied to both the list frame and the area frame replicate weights to compute variances.
Teacher replicates. The teacher replicate weights are generally equal to the school bootstrap replicate weights times the inverse of the conditional probability of selection of the teacher given the school was selected in the SASS school sample. These adjusted bootstrap replicate weights are provided on the file. BHR methodology was employed rather than bootstrap in two instances. First, if a school was selected with certainty and, subsequently, teachers were not sampled with certainty, no bootstrap replicate weights were available, so records were sorted by school stratum, order of selection, and control number, and then assigned variance stratum and panel.
The second instance was in the private area frame. These teacher sample records were assigned replicate weights by multiplying the school BHR replicate weights by the teacher's conditional probability of selection given the school was selected in the SASS school sample.
TFS teachers. Since the TFS sample was a proper subsample of the SASS teacher sample, the SASS teacher replicates were used for the TFS sample. The TFS basic weight for each TFS teacher was multiplied by each of the 48 SASS replicate weights divided by the SASS teacher full-sample intermediate weight for that teacher. To calculate 48 replicate weights which should be used for variance calculations, these TFS replicate basic weights were processed through the remainder of the TFS weighting system.
A variance estimate is obtained by first calculating the estimate for each replicate, then summing the squared deviations of the replicate estimates from the full-sample estimate, and finally dividing by the number of replicates:
When calculating variance estimates for some small subdomains of interest (e.g., vocational education teachers), sparseness of the data may result in there being no data from some replicates. This can result in either an extremely large variance estimate or failure of the software used to calculate the variance, with possibly a warning message.
WESTAT, Inc. has developed a PC-based replication program, WesVarPC. WesVarPC is available on the World Wide Web. The URL for WESTAT, Inc. is http://www.westat.com. There is a link on the WESTAT home page to the WesVarPC home page. WesVarPC version 2.1, along with the documentation, is available for download at no charge.
The purpose of the reinterview for the TFS was to evaluate response variance. Measuring response variance allows us to determine the degree of variability between the original interview responses and the reinterview responses. If the degree of variability is high, questions that need improvement can be determined. A sample of 1,545 cases was selected, expecting 1,000 completed reinterviews. Oversampling occurred to account for potential nonresponse based on the 1991-92 TFS. In actuality we obtained 870 completed reinterviews.
We used two reinterview questionnaires the TFS-3(R) for mail cases and the TFS-3(R)T for telephone cases. Each questionnaire contained a subset of questions from the original questionnaire.
The TFS reinterview took place from February 21, 1995 through June 16, 1995. An analysis of the reinterview data is in progress.
X. Confidentiality Protection Measures
The 1994-95 TFS data are released in accordance with the provisions of the General Education Provisions Act (GEPA) (20 USC 1221e-1) and the Carl D. Perkins Vocational Education Act. GEPA ensures privacy by ensuring that respondents will never be individually identified.
Under Public Law 100-297, the NCES is responsible for protecting the confidentiality of individual respondents and is releasing data to the public to use for statistical purposes only. Record matching or deductive disclosure by any user is prohibited.
To ensure that the confidentiality provisions contained in PL 100-297 have been fully implemented, procedures for disclosure avoidance were used in preparing the data tape in this release. Every effort has been made to provide the maximum research information consistent with reasonable confidentiality protections.
To prevent disclosure of the identities of teachers on the public use data tapes, state identifiers (for the public school teachers) and state, regional, and detailed affiliation and association codes (for the private school teachers) have been removed. In addition, continuous variables on the questionnaire that would permit disclosure of a teacher's identity (age and salary) have been coded into categories. The new categories for recoded variables are defined for the appropriate source codes on the attached tape record layouts. A few items have been deleted from the files altogether because of disclosure problems. These will be missing on the record layouts.
Difference between public and restricted use file. To protect the confidentiality of responding teachers, certain categories were collapsed on the public use file so that teachers cannot be identified. These included base academic year salary, teacher's age, total enrollment, percent minority enrollment, and the community type (rural, small town, urban, and central city) of the school. State identifiers and school affiliation were deleted from the public use file.
XI. Changes to TFS Content from 1991-92 to 1994-95
Some changes to wording and the order of specific items has occurred. Also, a new section was added to the Questionnaire for Current Teachers (TFS-3) to collect data on teaching methods. These new questions are Items 31-50.
XII. Caution Concerning the Measurement of Change Using 1991-92 and 1994-95 TFS
Changes in question wording. Caution must used in the interpretation of change estimates between 1991-92 and 1994-95 TFS since specific questions are not always worded the same in both surveys.
XIII. User Notes and Comments
We are interested in your reaction to the information presented here about the Teacher Followup Survey (TFS) data collection system as well as the microdata files we release. We welcome your recommendations for improving our survey work and data products. If you have suggestions or comments or want more information about this report, please contact:
Teacher Followup Survey
National Center for Education Statistics
1900 K Street NW, Suite 9000
Washington DC 20006
We are also interested in the research you do using the TFS data sets. We would be pleased to receive copies of reports, working papers, and published articles you write, which use data from the TFS. Send them to the address above.
FOOTNOTES:
[17] Kalton, G. and Kasprzyk, D. (1982), "Imputing for Missing Survey Responses," Proceedings of the Section on Survey Research Methods, American Statistical Association, 22-31; Kalton, G., Compensating for Missing Survey Data. Ann Arbor: Survey Research Center, University of Michigan, 1983; Kalton, G. and Kasprzyk, D. (1986), "The Treatment of Missing Survey Data," Survey Methodology, Vol. 12, No. 1, pp. 1-16; Little, R.J.A. and Rubin, D.B. (1987), Statistical Analysis with Missing Data, John Wiley and Sons; Madow, W.G., Olkin, I., and Rubin D.B. (eds.) 1983, Incomplete Data in Sample Surveys, Vols. 1, 2, and 3, New York, Academic Press.
[18] For more information about bootstrap variance methodology and how it applies to SASS and TFS, see: Efron, B (1982), The Jackknife, the Bootstrap and Other Resampling Plans, SIAM No. 38; Kaufman, S. (1992), "Balanced Half-sampled Replication with Aggregation Units," Proceedings of the Section on Survey Research Methods, American Statistical Association, 1992. Alexandria, VA: American Statistical Association. Kaufman, S. (1993), "A Bootstrap Variance Estimator for the Schools and Staffing Survey," Proceedings of the Section on Survey Research Methods, American Statistical Association, 1993. Alexandria, VA: American Statistical Association. "Properties of the Schools and Staffing Survey's Bootstrap Variance Estimator," Proceedings of the Section on Survey Research Methods, American Statistical Association, 1994. Alexandria, VA: American Statistical Association. Sitter, R.R. (1990), "Comparing Three Bootstrap Methods for Survey Data," Technical Report Series of the Laboratory for Research in Statistics and Probability, Carlton University.
Appendex D: SASS and TFS Data Products
