National Teacher and Principal Survey (NTPS)

5. DATA QUALITY AND COMPARABILITY

Sampling Error

Sampling errors arise when a sample of the population, rather than the whole population, is used to estimate some statistics. Different samples from the same population would likely produce somewhat different estimates of the statistic in question. This fact means that there is a degree of uncertainty associated with statistics estimated from a sample. This uncertainty is referred to as sampling variance and is usually expressed as the standard error of a statistic estimated from sample data. The approach used for calculating standard errors in the NTPS is jackknife replication. Jackknife replication methods involve dropping a small portion of the sample from the full sample and computing the statistic of interest for the retained and reweighted sample. The sum of squares of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic.

Nonsampling Error

Nonsampling error is a term used to describe variations in the estimates that may be caused by population coverage limitations, nonresponse bias, and measurement error, as well as data collection, processing, and reporting procedures. The sources of nonsampling errors are typically problems like unit and item nonresponse, differences in respondents' interpretations of the meaning of the survey questions, response differences related to the particular time the survey was conducted, and mistakes in data preparation.

Unit nonresponse. Unit nonresponse error results from nonparticipation of schools, teachers, and principals. NCES standards require base‑weighted response rates greater than 85 percent. The weighted response rates are displayed in Table NTPS‑1 for the data collection of 2016 and 2018.

For each of the three populations of interest to the NTPS (schools, principals, and teachers), nonresponse bias analyses were carried out. First, the base‑weighted distribution of responding members was compared to the base‑weighted distribution of sampled members through t tests to identify any potential bias prior to weighting adjustments. Next, weighting adjustments were designed to reduce or eliminate nonresponse bias and to reduce the variance introduced due to sampling by adjusting the sample estimates to known totals from the frame. Following weighting adjustment, evidence of potential bias remained for schools, principals, and teachers. For more details regarding the nonresponse bias analysis, see the Survey Documentation for the 2017–18 National Teacher and Principal Survey (Cox et al. forthcoming).

Item nonresponse. The item response rate indicates the percentage of respondents who answered a given survey question. The weighted NTPS item response rate is calculated by dividing the weight number of respondents who provided an answer to an item by the weighted number of respondents who were eligible to answer that item.

Data Comparability
The NTPS is a new survey that is strongly based on the SASS. However, care must be taken in estimating changes over time in data elements that both surveys have in common because some of the change measured may not be attributable to a change in the education system.

Some of the change may be due to changes in the sampling frame, changes in the questionnaire item wording, or other changes. Additionally, the NTPS is a different survey than the SASS and pulls data from a larger variety of sources and timeframes than the SASS did. While the SASS collected data on student race/ethnicity, special programs, and high school graduations, the 2015–16 NTPS gets this information from external sources. Data on student gender and race/ethnicity are taken from the 2014–15 CCD, while graduation rates come from the 2014–15 EDFacts data and information on special programs come from the 2013–14 Civil Rights Data Collection (CRDC).

Additionally, the 2015–16 NTPS is not representative at the state level, and comparison to the SASS may only be made at the national level. Private sector schools are also excluded from the 2015–16 NTPS.

Furthermore, the 2017–18 NTPS collected and reported information on the National School Lunch Program differently than previous years of both NTPS and SASS. In this cycle, schools were asked to report the percentage of K-12 students approved for the program. Additionally, schools that did not participate in the program used to be treated as a separate category for reporting purposes but, in this cycle, are grouped with schools that participated in the program but had no students approved for the program. This decision was based on the small size of the number of schools that did not participate and the categories used for sampling, calculating response rates, and conducting bias analyses. Due to both the change in the question and the change in categorization of non-participating schools, users should exercise caution when comparing estimates for, or reported by, the percentage of students approved for free and reduced-price lunches.

Table NTPS‑1. NTPS weighted unit level response rates: 2016 and 2018
Year	Public School	Public School Principal	Public School Teacher	Private School	Private School Principal	Private School Teacher
2016	72.5	71.8	57.2¹	†	†	†
2018	72.5	70.2	67.0²	64.5	62.6	53.9³
†Not applicable. ¹Represents the overall response rate for public school teachers in the 2015–2016 cycle, calculated as the product of the response rate to two stages: the teacher listing form (84.4%) and the teacher questionnaire (67.8%). ²Represents the overall response rate for public school teachers in the 2017–2018 cycle, calculated as the product of the response rate to two stages: the teacher listing form (87.1%) and the teacher questionnaire (76.9%). ³Represents the overall response rate for private school teachers in the 2017–2018 cycle, calculated as the product of the response rate to two stages: the teacher listing form (71.0%) and the teacher questionnaire (75.9%). SOURCE: Methodology reports for the National Teacher and Principal Survey. Reports are available at https://nces.ed.gov/pubsearch/getpubcats.asp?sid=122.

Top