Private School Universe Survey (PSS)



4. SURVEY DESIGN

Target Population


The target population for PSS is all schools in the United States that meet the NCES definition of private schools (i.e., not supported primarily by public funds, providing classroom instruction for one or more of grades kindergarten through 12 or comparable grade levels, and having one or more teachers). The PSS universe consists of a diverse population of schools. It includes both schools with a religious orientation (e.g., Catholic, Lutheran, or Jewish) and nonsectarian schools, with programs ranging from regular to special emphasis and special education.

Sample Design

NCES uses a dual-frame approach for building its private school universe. The primary source of the PSS universe is a list frame containing most private schools in the country. The list frame is supplemented by an area frame, which contains additional schools identified during a search of randomly selected geographic areas around the country. The two frames are used together to estimate the population of private schools in the United States.

List frame. In an effort to ensure a complete population list of all private elementary and secondary schools in the United States, NCES updates the list frame every two years in preparation for the next PSS administration. The list frame was initially developed for the 1989–90 survey. The list is updated periodically by matching it with lists provided by nationwide private school associations, state departments of education, and other national private school guides and sources.

The basis of the current survey’s list frame is the previous PSS. In order to expand coverage to include private schools founded since the previous survey, NCES requests lists of schools from the 50 states and the District of Columbia in advance of each survey administration. Requests are made to state education departments, as well as to other departments, such as state departments of health or recreation. NCES also collects membership lists from about 30 private school associations and religious denominations. Schools on the state and association lists are compared to the base list, and any school not matching a school on the base list is added to the universe list.

The list-building component is the primary means for improving coverage of private schools. The basis for the 2017–18 PSS list frame was the 25,224 private schools from the 2015–16 PSS list frame and the certainty portion of the area frame. In order to provide coverage of private schools founded since 2015 and to improve coverage of private schools existing in 2017, NCES received and processed membership lists from 29 private school associations and religious denominations. The departments of education from 50 states and the District of Columbia provided lists of private schools. Additional private schools that contacted NCES to be included in the PSS were added as list frame cases. Schools on private school association membership lists and the state lists were compared to the base list, and any school that did not match a school on the base list was added to the universe list. A total of 40,563 schools were included in the 2017–18 list frame.

Area frame. The list frame is supplemented by an area frame, which contains additional private schools identified during a search of telephone books and other sources in randomly selected geographic areas around the country. The area frame search is conducted by the Census Bureau. Each area’s list is created from a set of predetermined sources within that area and then matched against the updated list frame universe to identify schools missing from the updated list frame.

The United States is divided into 2,062 primary sampling units (PSUs), each consisting of a single county, independent city, or cluster of geographically contiguous areas. The eight PSUs with private school enrollment in the 2010 census populations greater than 1.7 million were selected with certainty for the private school survey. In addition to these certainty PSUs, the area frame consists of a sample of PSUs selected independently from the previous PSS sample. A minimum of two PSUs are allocated to each of the 16 strata, which are defined by (1) four Census regions (Northeast, Midwest, South, or West); (2) metro/nonmetro status (two levels); and (3) whether the PSU’s percentage of private school enrollment exceeds the median percentage of private enrollment of the other PSUs in the census region/metro status strata (two levels). Within a stratum, the sample PSUs are selected with probability proportional to the square root of the Census 2000 PSU private school enrollment.

A total of 124 distinct PSUs (160 counties) were in the 2017–18 PSS area frame sample. Within each of these PSUs, the Census Bureau attempted to find all eligible private schools. A block-byblock listing of all private schools in a sample of PSUs was not attempted. Rather, National Processing Center staff created the frame by using such sources as online yellow pages (yp.com), local Catholic dioceses, local education agencies, and local government offices. The Census Bureau also obtained a list of eligible private schools for area frame sample PSUs from the InfoUSA database (see http://www.infousa.com/.) Once the area search lists were constructed, they were matched with the NCES private school universe list. Schools that did match the universe list were deleted from the area frame. A total of 2,821 schools were added to the universe from the area frame in 2017–18.

Due to differences in methodology and definitions, the results of the 1993–94 and subsequent area search frames are not strictly comparable to the results of earlier years. Prior to 1993, an initial eligibility screening was performed by telephone for area frame schools before the questionnaire was mailed out. Ineligible schools were declared out of scope at that time, and eligible schools were either interviewed by telephone or sent a questionnaire. In the 1993–94 PSS, screener questions were added to the survey instrument to determine eligibility. Ineligible schools were not eliminated until the questionnaires were returned.

Top

Data Collection and Processing

The U.S. Census Bureau is the collection agent for PSS. The 2017–18 PSS data were collected between October 2017 and August 2018. The 2017–18 PSS data are for the 2017–18 school year except the high school graduate data, which are for the 2016–17 school year.

Reference dates. The official reference date for reporting PSS information is October 1.

Data collection. The data collection for the 2017–18 PSS coincided with the data collection phase of the private school component of the 2017–18 National Teacher and Principal Survey (NTPS). After the NTPS data collection, PSS records were created for the private schools responding to NTPS from the NTPS data and were merged into the PSS. For the non-NTPS cases the List Frame followed two data collection tracks: non-Amish and non-Mennonite schools that were offered an internet response option and schools that do not use internet or telephones (primarily Amish and Mennonite schools) that never receive any reference to an internet response option. The 2017–18 PSS data collection began on October 2, 2017, with the internet instrument deployment, and the mailout of a letter to non-Amish and non-Mennonite schools describing how to respond to the 2017–18 PSS online with a User ID and password. For Amish and Mennonite schools, this first mailing included information about the PSS and their first paper questionnaire with a return envelope, there was no internet insert letter.

On October 16th, the second mail out occurred. Here, non-Amish and non-Mennonite schools received a paper questionnaire and Amish and Mennonite schools received their second paper questionnaire. Reminder postcards were mailed out to nonresponding Amish and Mennonite schools on October 30, 2017, and to nonresponding non-Amish and non-Mennonite schools on November 13, 2017.

Because Amish and Mennonite schools do not often possess telephones, a field nonresponse follow-up effort targeting these schools ran from January 8 to February 23, 2018. Follow-up of nonresponding non-Amish and non-Mennonite schools was conducted first by computer-assisted telephone interviewing (CATI) February 5 to March 16, 2018. Both nonresponding list frame and nonresponding area frame cases were included in this CATI operation. Finally, from April 9 through May 23, 2018, a field nonresponse follow-up operation was conducted for cases that did not respond during the CATI operation, targeting all list frame and area frame nonresponse cases. The online option was kept open throughout data collection. Data collection for the 2017–18 PSS ended on August 17, 2018.

Editing. The 2015–16 PSS data were captured using a combination of manual data keying and imaging technology, using the Integrated Computer Assisted Data Entry (iCADE) system. The questionnaires were disassembled and each duplex page was scanned. Clerical staff reviewed and resolved data issues due to system errors such as the inability to read a barcode, inability to recognize a page corner point, or checkbox ambiguities (e.g., marks outside checkbox, scratch outs, random marks on a page). Each imaged response was presented to a keying operator and all fields except the checkboxes were keyed from image (KFI). The checkboxes were captured using Optical Mark Recognition (OMR). All non-blank data fields were 100 percent KFI verified.

For data collected during the telephone follow-up phase, preliminary quality assurance and editing checks take place at the time of the interview. The data collection instrument is designed to alert interviewers to inconsistencies reported by the respondent so that any necessary corrections can be made at this time. Data from the CATI facilities are transmitted to Census headquarters for further processing where they undergo extensive editing, including:

  • Pre-edits by Census Bureau staff to correct unacceptable entries using information reported in other questionnaire items or information from other sources;
  • Consistency edits to compare data in different fields for consistency;
  • Blanking edits to verify that skip patterns on the questionnaire were followed; and
  • Logic edits used to fill the items with missing values using previous collection data and to correct some inconsistencies between students by race and total enrollment using the ratio adjustment.

Top

Estimation Methods

Weighting adjusts the number of schools in the area frame sample up to fully represent the number of schools missing from the list frame, and adjusts the survey data from both the area and list components for school nonresponse. Imputation is used to compensate for item nonresponse.

Weighting. PSS data from the area frame component are weighted to reflect the sampling rates (probability of selection) in the PSUs. Survey data from both the list- and area-frame components are adjusted for school nonresponse. This represents a departure from procedures used in the 1989–90 survey, which adjusted for total nonresponse (i.e., school nonresponse) and for partial nonresponse associated with four specific PSS data elements. Since 1991, only one weight has been required, due to a complex imputation process used to compensate for item nonresponse. When estimates are produced for schools and other data elements, the same PSS school weight should be used. A brief description of the components comprising the PSS weight follows:

Wi, the PSS weight for all data items for the ith school, is

Wi = BWi x NRc

where

BWi is the base weight, or the inverse of the selection probability for school i (BWi = 1 for all list frame schools; BWi = the inverse of the PSU probability of selection for area frame schools), and

NRc is the nonresponse adjustment factor, or weighted ratio of the sum of the in-scope schools to the sum of the in-scope responding schools in cell c, using BWi as the weight.

The cells used to compute the nonresponse adjustment are defined differently for list-frame and area-frame schools. In the 2015–16 PSS, for schools in the list frame, the cells were defined by affiliation, locale type, grade level, Census region, and enrollment. The nonresponse adjustment cells for area-frame schools were defined by three-level typology (Catholic, Other religious, Nonsectarian), and grade level.

If the number of schools in a cell was fewer than 15 or the nonresponse adjustment factor was greater than 1.5, then that cell was collapsed into a similar cell. The cells for traditional schools from the list frame were collapsed within enrollment category, locale type, grade level, and Census region. Cells for k-terminal schools from the list frame were collapsed within enrollment category, locale type, Census region (if applicable), and affiliation. Cells for traditional schools from the area frame were collapsed within grade level and then within three-level typology. Cells for k-terminal schools from the area frame were collapsed within three-level typology.

Imputation. Since the 1991–92 PSS, imputation has been used to compensate for item nonresponse in records classified as interviews (i.e., required items are completed). All items that are missing data after the editing process are imputed. The first survey, the 1989–90 PSS, used weighting adjustments for both interviews and noninterviews.

After the edit and final interview status processing are complete, if there are missing values within some records classified as interviews, values are first imputed to the missing data using donor imputation. In donor imputation, values are created by extracting data from the record for a sample case with similar characteristics, using a hot-deck imputation methodology.

After donor imputation, for records that still have missing values for some items, entries are clerically imputed during analyst imputation. Staff reviews the data record, sample file record, and the questionnaire and identify a value consistent with the information from those sources. This procedure is used when: (1) no suitable donor is found, (2) the computer method produces an imputed entry that is unacceptable, and (3) the nature of the item requires an actual review of the data rather than a computer-generated value.

Top

Recent Changes

Several changes to the questionnaire have been introduced in the previous PSS cycles. In the 1993–94 PSS, three major revisions were made. First, a new design was implemented to facilitate respondent reporting by clearly indicating skip patterns through the use of arrows as well as words and by minimizing the number of questions asked on each page. Second, content on kindergarten programs was expanded to collect the type of kindergarten program in addition to the kindergarten student and teacher counts requested in earlier surveys. Third, data on the racial/ethnic makeup of the school’s student body were collected for the first time.

Modifications made to the 1995–96 PSS included adding nursery and prekindergarten, transitional kindergarten, and transitional first-grade enrollment counts to the enrollment item. Questions on the length of the school day and number of days per week for kindergarten, transitional kindergarten, and transitional first grade were also added. “Early childhood program/day care center” was added as a category for type of school. The 1993–94 PSS questionnaire items concerning types of prekindergarten programs and the number of prekindergarten teachers were deleted.

In the 1997–98 PSS, the following items were added to the survey instrument: (1) whether or not the school is coeducational (if yes, the number of male students; if no, whether the school is all female or all male); and (2) whether or not the school has a library or library media center.

There were few changes in the 1999–2000 PSS. One religious affiliation—Church of God in Christ—was added, and several associations were added: Association of Christian Teachers and Schools, National Coalition of Girls’ Schools, and state or regional independent school associations. The item that previously collected data on the number of graduates that applied to 2–year or 4–year colleges was changed to collect data on the percentage of graduates who went on to attend three types of schools: 2–year colleges, 4–year colleges, and technical or other specialized schools. There also was a minor change in the definition of community type. Beginning with the 1999–2000 PSS, schools that were “rural within a Metropolitan Statistical Area” were included in the “Rural/small town” community type, while prior to the 1999–2000 PSS they were included in the “Urban fringe/large town” community type.

The 2001–02 PSS questionnaire content was relatively unchanged from the 1999–2000. One question was added to item 2 (the screener item)—“Is the school named on the front of this questionnaire located in the United States?” This question was added to facilitate the exclusion of schools from the PSS that were located outside of the United States, but had been added during the list building or area search because the school had an office with an address in the United States.

Additionally, in order to test the feasibility and benefits of collecting PSS data over the Internet, the 2001–02 PSS included an Internet response option test. The final response rate for Internet submissions was 15.4 percent for schools that received the option (5.1 percent of all schools).

Changes made to the 2003–04 PSS were minor and involved frame creation methodology, data collection procedures, and weighting procedures. For example, whereas in the 2001–02 PSS, the base weight for area frame schools was equal to the inverse of the probability of selecting the PSU in which the school resided, in the 2003–04 PSS, the base weight for area frame schools also contained a nonunitary subsampling factor for schools named solely in non-Roman Catholic religious institution lists.

Caution, however, should be used in comparing 2003–04 PSS community type estimates to those of previous years. Although the definition of community type remained unchanged, the 2003–04 PSS community types are based on the Consolidated Statistical Area/Core-Based Statistical Area rather than on the Standard Metropolitan Statistical Area/Metropolitan Statistical Area, which was used prior to the 2003–04 PSS. Also, community type is based on 2000 census data; prior to the 2003–04 PSS, community type was based on 1990 census information.

There were few changes in the 2005–06 PSS. One religious affiliation—Church of the Nazarene—was added. Also, the 2005–06 PSS used the new 12-level urban-centric locale codes, rather than the 8-level locale codes based on the Core-Based Statistical Area.

There was one change in the 2007–08 PSS. In 2005–06, non-Roman Catholic religious institutions were contacted during the area-frame operation while in 2007–08 they were not.

There were some changes to the 2009–10 PSS from the 2007–08 PSS. In 2007–08, because PSS coincided with the data collection phase of the private school component of the 2007–08 SASS, the private schools selected for SASS were excluded from the PSS, and the schools selected for SASS received a SASS private school questionnaire only, while the remaining private schools were sent a PSS questionnaire. The PSS questionnaire used the same wording as the SASS questionnaire, but contained only a subset of the SASS questionnaire items. After data collection, the data for the SASS cases were merged into the PSS universe. In 2009–10, PSS data collection did not coincide with the SASS, and questionnaire was sent to all private schools. Additionally, in 2007–08, telephone follow-up was conducted from one Census Bureau CATI facility, while in 2009–10, telephone follow-up was conducted from two Census Bureau CATI facilities. The 2007–08 PSS did not offer the online version of the questionnaire, but for the 2009–10 PSS, respondents were given the option during the mailout period and telephone follow-up of completing the questionnaire online instead of completing the paper questionnaire.

There was also one change in PSS frame creation methodology between 2007–08 and 2009–10. In 2007–08, half of the noncertainty PSUs in the area frame were selected independently from the 2005–06 area frame PSUs and half overlapped with the 2005–06 area frame. This overlapping was not done for 2009–10 since it was determined the overlapping did not serve to reduce the variance of change estimates appreciably. Therefore, in the 2009–10 survey, all 116 noncertainty PSUs were sampled independently from the 2007–08 survey PSUs.

While the 2009–10 questionnaire content was relatively unchanged from that of 2007–08, there were some changes to race categories and four associations were deleted from the private school association list. There were no changes to the questionnaires in 2011–12, 2013–14, and 2015–16.

Changes made to the 2015–16 PSS involved frame creation methodology and data collection procedures. Before the 2015–16 data collection, the Area Search Frame (ASF) creation was performed by the field staff. Field offices would research the PSU's that fell in their regions. In the 2015–16 PSS, this effort was centralized at the National Processing Center (NPC) as a clerical operation. All activities were the same except for the fact that physical yellow page phone books were replaced with online yellow page resources, primarily yp.com. Changes in 2015–16 also included conducting a branding experiment and including endorsements from religious or educational affiliations in the letters mailed to schools.

Future Plans

The PSS will continue as a biennial survey.

Top