- Introduction
- Adult Literacy and Lifeskills Survey (ALL)
- Academic Libraries Survey (ALS)
- Baccalaureate and Beyond (B&B) Longitudinal Study
- Beginning Postsecondary Students (BPS) Longitudinal Study
- Beginning Teacher Longitudinal Study(BTLS)
- Common Core of Data (CCD)
- Current Population Survey (CPS)
- Early Childhood Longitudinal Study, Birth Cohort (ECLS-B)
- Early Childhood Longitudinal Study, Kindergarten Class of 1998‑99 (ECLS‑K)
- Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS‑K:2011)
- Educational Longitudinal Study of 2002 (ELS 2002)
- Quick Response Information System (FRSS/PEQIS)
- High School and Beyond (HS&B) Longitudinal Study
- High School Longitudinal Study of 2009 (HSLS:09)
- High School Transcript Studies (HSTS)
- International Adult Literacy Survey (IALS)
- Integrated Postsecondary Education Data System (IPEDS)
- Middle Grades Longitudinal Study of 2017– 18 (MGLS:2017)
- National Assessment of Adulty Literacy (NAAL)
- National Assessment of Educational Progress (NAEP)
- National Adult Literacy Survey (NALS)
- National Education Longitudinal Study of 1988 (NELS 88)
- National Household Education Survey Program (NHES)
- National Longitudinal Study of the High School Class of 1982 (NLS 72)
- National Postsecondary Student Aid Study (NPSAS)
- National Study of Postsecondary Faculty (NSOPF)
- National Teacher and Principal Survey (NTPS)
- Program for the International Assessment of Adult Competencies (PIAAC)
- Progress In International Reading Literacy Study (PIRLS)
- Program For International Student Assessment (PISA)
- Private School Universe Survey (PSS)
- Overview
- Uses of Data
- Key Concepts
- Survey Design
- Data Quality and Comparability
- Contact Information
- Methodology and Evaluation Reports
- PSS Website
- Download PDF
- School and Staffing Survey (SASS)
- SASS Principal Follow-Up Survey (PFS)
- SASS School Library Media Center Survey (SLS)
- SASS Teacher Follow-up Survey (TFS)
- Survey of Earned Doctorates (SED)
- Crime and Safety Surveys (SCS & SSOCS)
- Teaching and Learning International Survey (TALIS)
- Trends In International Mathematics and Science Study (TIMSS)
Private School Universe Survey (PSS)
5. DATA QUALITY AND COMPARABILITY
Sampling Error
Only the area frame contributes to the standard error in the PSS. The list frame component of the standard error is always 0. Estimates of standard errors are computed using half-sample replication.
Because the area frame sample of PSUs is small (124 out of a total of 2,062 eligible PSUs), there is a potential for unstable estimates of standard errors. This is particularly true when the domain of interest is small and there may not be enough information to compute a standard error. Stabilizing the standard error estimate given the level of detail of the PSS estimates would require a much larger PSU sample. The current area frame is designed to produce regional estimates.
Nonsampling Error
Coverage error. Undercoverage in the list and area frames is one possible source of nonsampling error. Because the PSS uses a dual-frame approach, it is possible to estimate the coverage or completeness of the PSS. A capture-recapture methodology is used to estimate the number of private schools in the United States and to estimate the coverage of private schools. In the 2003–04 PSS, the conservative coverage rate for traditional private schools was 96 percent; for k-terminal private schools, it was 85 percent. In the 2005–06 PSS, the overall coverage rate was 98 percent. In the 2007–08 PSS, the conservative coverage rate for traditional private schools was 96 percent; for k-terminal private schools, it was 93 percent. In the 2009–10 PSS, the conservative coverage rate for traditional private schools was 95.8 percent; for k-terminal private schools, it was 85.5 percent. In the 2011–12 PSS, the conservative coverage rate for traditional private schools was 98.8 percent; for k-terminal private schools, it was 98.2 percent. In the 2013–14 PSS, the conservative coverage rate for traditional private schools was 96.3 percent; for k-terminal private schools, it was 89.5 percent. In the 2015–16 PSS, the conservative coverage rate for traditional private schools was 89.1 percent; for k-terminal private schools, it was 76.5 percent.
A study comparing the quality of PSS frame coverage to that of the commercial Quality Education Data database of schools is discussed in Lee, Burke, and Rust (2000).
Nonresponse error. There are two types of nonresponse error: unit nonresponse and item nonresponse.
Unit nonresponse. In the 2017–18, 2015–16, 2013–14, 2011–12, 2009–10, 2007–08, and 2005–06 PSS, the survey data from the area frame component were weighted to reflect the sampling rates (probability of selection) of the PSUs. Survey data from both the list and area frame components were adjusted for school nonresponse. For 2017–18 there were 22,895 completed interviews and 5,217 noninterviews. After weighting the area frame component, there were 24,873 interviews and 7,588 noninterviews, resulting in a response rate of 76.6 percent. For 2015–16 there were 22,428 completed interviews and 7,207 noninterviews. After weighting the area frame component, there were 23,938 interviews, resulting in a response rate of 69.2 percent. For 2013–14 there were 24,566 completed interviews and 5,073 noninterviews. After weighting the area frame component, there were 27,110 interviews, resulting in a response rate of 80.6 percent. In 2011–12 there were 26,983 completed interviews and 2,312 noninterviews. After weighting the area frame component, there were 28,345 interviews, resulting in a response rate of 91.8 percent. In the 2009–10 PSS, there were 28,217 interviews and 1,856 noninterviews. After weighting the area frame component, there were 31,221 interviews and 2,148 noninterviews, resulting in a response rate of 93.6 percent. In the 2007–08 PSS, there were 28,450 interviews and 2,527 cases that were noninterviews. After weighting the area frame component, these became 30,748 interviews and 2,992 noninterviews—the weighted response rate was 91 percent. In the 2005–06 PSS, there were 29,784 interviews and 1,867 cases that were noninterviews. After weighting the area frame component, these became 32,865 interviews and 2,159 noninterviews—the weighted response rate was 94 percent. In the 2003–04 PSS, of the 41,184 schools included (both traditional and k-terminal), some 9,336 cases were considered out-of-scope (that is, not eligible for the PSS). A total of 30,071 private schools completed a PSS interview, while 1,777 schools refused to participate, resulting in an overall unweighted response rate of 94 percent. When the area-frame schools were weighted by the inverse of the probability of selection, the weighted response rate was 94 percent as well. In the 2001–02 PSS, the weighted response rate for traditional schools was 95 percent (96 percent unweighted); for k-terminal schools, the response rates were 97 and 96 percent, respectively. In 1999–2000, both the weighted and unweighted response rates were 93 percent for traditional schools; they were 99 and 98 percent, respectively, for k-terminal schools.
Item nonresponse. Weighted item-level response rates in the 2015–16 PSS were generally high, ranging from 89 to 100 percent. Because the weighted item response rates were all greater than 85 percent, no item-level bias analyses were performed. Weighted item-level response rates in the 2011–12 PSS were generally high, ranging from 96 to 100 percent. Because the weighted item response rates were all greater than 95 percent, no item-level bias analyses were performed. In the 2009–10 PSS, all of the weighted response rates were greater than 85 percent. In the 2007–08 PSS, all of the weighted response rates were greater than 85 percent. The weighted item response rates for all but one variable—the percentage of graduates who went to 2–year colleges—were greater than 85 percent in 2005–06. In the 2003–04 PSS, all of the weighted response rates were greater than 85 percent. In the 2001–02 PSS, for traditional schools, all but three items had weighted response rates greater than 90 percent. The three lower rates (ranging from 77.5 percent to 86.3 percent) pertained to the percentage of graduates who went to 4–year colleges, 2–year colleges, and technical or other specialized schools. Values for items with missing data are imputed to compensate for item nonresponse.
Measurement error. NCES seeks to minimize measurement error by developing survey content in consultation with representatives of private school associations, reviewing the questionnaire and instructions extensively before distribution, requiring that the data that are not scanned are 100 percent key-verified, and processing the survey data through a comprehensive series of edits to verify accuracy and consistency.
Intersurvey Consistency in NCES Private School Surveys
The PSS and the private school component of Schools and Staffing Survey (SASS) were fielded in the same school year for the first time in 1993–94. Even though these two surveys measured some of the same variables (schools, teachers, and students), the 1993–94 results were not in agreement due to sampling and other errors. PSS results are likely to be the more accurate since the PSS served as the sampling frame for the SASS private school component (a sample of around 3,000 schools). Special methodological studies of these two surveys have been done, including comparisons among statistical and computational procedures aimed at achieving consistency between the estimates of private schools, private school teachers, and private school students in the 1993–94 PSS and in the 1993–94 SASS—see Scheuren and Li (1995, 1996). The last SASS data collection took place during the 2011–12 school year and has been replaced with the National Teacher and Principal Survey (NTPS). The 2015–16 NTPS sample did not include private schools. Private schools were included in the 2017–18 NTPS.
Data Comparability
While changes to survey design and content generally result in improved data quality, they also impact the comparability of data over time. Recent changes to the PSS and to the comparability of PSS data (both within the PSS itself and with other data sources) are discussed below.
Design change. Changes in the survey design of the 1995–96 PSS resulted in an increased number of private schools in the survey population. First, seven new association lists were obtained, adding 512 new schools to the list frame. In previous years, the area frame was relied upon to include these schools. Second, the area search results were not strictly comparable to those in previous years due to procedural differences. The 1995–96 PSS was the first survey to verify the control of schools marked as public in the screener item. Final determination of school control was based on a review of the school’s name and other identifying information. As a result, several schools that had been marked as public (but which were obviously private) were added back into the PSS. They were counted as interviews if the required data were provided or as noninterviews if the required data were missing. Third, the eligibility criteria for the PSS were changed to no longer require schools to have 160 days in the school year or to conduct classes for at least 4 hours per day. Fourth, the PSS definition of a school was expanded to include programs where kindergarten is the highest grade (k-terminal schools). Additional lists of programs that might have a kindergarten were requested from nontraditional sources, and the area search was expanded to search for programs with a kindergarten. Some schools meeting the traditional PSS definition of a school (any of grades 1–12 or comparable ungraded levels) were discovered in these lists. When added to the PSS, these schools also increased the estimates of traditional schools.
Note that even when the population of schools is about the same from one survey to the next, it may represent a different set of schools. For example, the number of schools was around 27,000 in both 1997–98 and 1999–2000, although about 1,700 schools were added to the PSS universe in 1999–2000. This suggests that a nearly equal number of schools dropped out of the universe between 1997–98 and 1999–2000. Comparisons of the 1999–2000 PSS private school estimates with those from the 2001–02 PSS, however, show an overall increase in the number of private schools between 1999–2000 and 2001–02 (to about 29,000).
Questionnaire changes. Several modifications have been made to the format and content of the PSS questionnaire since 1991–92. A number of items were added (including race/ethnicity of students), and some items were deleted or modified.
Comparisons within the PSS. The estimated number of schools in 2015–16 was not statistically different from that of 2013–14. The estimated number of private school students and full-time equivalent (FTE) teachers increased between 2013–14 and 2015–16. The estimated number of schools and students in 2013–14 was not statistically different from that of 2011–12. The estimated number of private FTE teachers increased between 2011–12 and 2013–14. The estimated number of schools in 2011–12 was not statistically different from that of 2009–10. The estimated number of students and FTE teachers decreased between 2009–10 and 2011–12. The estimated number of schools in 2009–10 was not statistically different from that of 2007–08. The estimated number of private school students and FTE teachers decreased between 2007–08 and 2009–10 (by 372,332 students and 18,852 teachers). The estimated number of schools decreased between 2005–06 and 2007–08 (by 1,314 schools). The estimated number of private school students and FTE teachers in 2007–08 were not statistically different from those of 2005–06. The estimated number of private schools and students decreased between the 2001–02 and 2003–04 PSS data collections (by 889 schools and 218,741 students). The estimated number of FTE teachers in 2003–04 was not statistically different from that in 2001–02. Comparisons of the 2001–02 PSS estimates with those from previous PSS data collections show increases in the number of private schools, students, and teachers between 1999–2000 and 2001–02. Comparisons of the 1999–2000 PSS estimates with those from previous surveys show no significant change in the estimated number of private schools; however, they do indicate an increase in the estimated number of private school teachers and students.
Comparisons with the Current Population Survey. A comparison of the PSS estimate of K–12 students enrolled in all private schools (traditional and k-terminal) with the household survey estimate from the 2015 October Supplement of the Current Population Survey (CPS) (U.S. Census Bureau 2016) shows that the PSS estimate of 4,903,596 does significantly differ from the CPS estimate of the number of private school students in grades K–12 in October 2015 of 4,700,000. For 2013, a comparison showed that the PSS estimate of 4,576,410 does significantly differ from the CPS estimate of the number of private school students in grades K–12 in October 2013 of 4,500,000. For 2011–12, the comparison showed that the PSS estimate of 4,494,845 differs significantly from the CPS estimate of 4,400,000. For 2009–10, the comparison showed that the PSS estimate of 4,700,119 does significantly differ from the CPS estimate of 4,500,000 private school students in grades K–12 in October 2009. In 2007–08, the CPS estimate showed that that the PSS estimate of 5,072,451 did significantly differ from the CPS estimate of the number of private school students in grades kindergarten through 12 in October 2007 of 4,817,000. A comparison of the 2003–04 PSS estimate of K–12 students enrolled in all private schools (traditional and k-terminal) with the household survey estimate from the 2003 October Supplement to the CPS showed that the PSS estimate of 5,212,992 students was not statistically different from the CPS estimate of 5,259,000 students. A comparison of the 2001–02 PSS estimate of K–12 students enrolled in all private schools (traditional and k-terminal) with the household survey estimate from the October 2001 CPS shows that the PSS estimate of 5,439,925 is higher than the CPS estimate of 5,164,000; the 95 percent confidence interval of the PSS estimate ranges from 5,383,898 to 5,495,952 students, while that of the CPS estimate ranges from 4,956,000 to 5,372,000 students. In the 1995–96 school year, the PSS and CPS estimates did not differ significantly; in 1997–98, the PSS estimate was higher than the CPS estimate; and, in 1999–2000, the PSS estimate was lower than the CPS estimate. Comparisons between CPS and PSS enrollment estimates for earlier years are not as informative since, prior to 1995–96, the PSS estimates did not include the kindergarten enrollment from k terminal schools, whereas the CPS has always included it.
Comparisons with National Catholic Educational Association data. Comparisons of the PSS estimates for Catholic schools, students, and full-time equivalent (FTE) teachers (traditional schools) with NCEA 2016 data for the 2015–16 school year show differences in the FTE teacher counts (151,101 versus 143,186) between the two data sources. Comparisons of the PSS estimates for Catholic schools, students, and full-time equivalent (FTE) teachers (traditional schools) with NCEA 2014 data for the 2013–14 school year show differences in the FTE teacher counts (154,316 versus 138,992) between the two data sources.
For the 2011–12 school year, comparisons of the PSS estimate for Catholic schools, students, and full-time equivalent (FTE) teachers with the National Catholic Educational Association (NCEA) 2011 data for the 2011–12 school year show differences in the FTE teacher counts (138,071 versus 151,395) between PSS and NCEA, respectively. The survey methodologies used by NCES and NCEA are different; while NCES surveys private schools directly, NCEA surveys archdiocesan and diocesan offices of education and some state Catholic conferences. The NCEA and PSS computations of FTEs differ in the weight assigned to part-time teachers, thus the FTE teacher counts are not strictly comparable between PSS and NCEA. For the 2009–10 school year, comparisons of the PSS estimates for Catholic schools, students, and FTE teachers with the NCEA 2010 data show differences in the student (1,969,079 versus 2,009,641) and FTE teacher counts (154,316 versus 142,620). For the 2007–08 school year, comparisons of the PSS estimates for Catholic schools, students, and FTE teachers with the NCEA 2008 data show differences in the school (7,507 versus 7,378), student (2,156,173 versus 2,270,913) and FTE teacher counts (146,627 versus 160,075) between PSS and NCEA, respectively. Comparisons of the PSS estimates for Catholic schools, students, and FTE teachers with the NCEA data for the 2003–04 school year show differences in the number of students (2,365,220 vs. 2,484,252) and FTE teachers (152,611 vs. 162,337) between PSS and NCEA, respectively. The difference between the PSS estimate of 7,919 Catholic schools and the NCEA count of 7,955 schools is not statistically significant.
For the 2001–02 school year, comparisons of the PSS estimate for Catholic schools with the NCEA data show differences in the school and student counts. The NCEA count of 8,000 schools is below the lower limit of the 95 percent confidence interval of the PSS estimate of Catholic schools (which ranges from 8,112 to 8,302). The NCEA K–12 student count of 2,553,277 is higher than the upper limit of the 95 percent confidence interval of the PSS estimate of Catholic students (which ranges from 2,492,773 to 2,538,274). Both the NCEA teacher count of 163,004 and the PSS estimate of 155,514 include part- and full-time teachers in the computation of full-time equivalents (the 95 percent confidence interval of the PSS estimate ranges from 153,902 to 157,126).
NCES publication criteria for the PSS. NCES criteria for the publication of an estimate are dependent on the type of survey—sample or universe. The PSS includes both types of surveys: (1) a sample survey of PSUs (area frame) that collects data on schools not in the list frame (the number of PSUs changes for each administration); and (2) a complete census of schools belonging to the list frame. NCES has established a rule that published PSS estimates must be based on at least 15 schools. If the estimate satisfies this criterion and the coefficient of variation (the standard error divided by the estimate) is between 30 and 50 percent, the estimate is identified as having a large coefficient of variation and the reader is referred to a table of standard errors. If the coefficient of variation is greater than 50 percent, the estimate is suppressed.