Appendix B—Technical Notes

Background

The Private School Universe Survey (PSS) is conducted by the National Center for Education Statistics (NCES) of the United States Department of Education in order to collect basic information on American private elementary and secondary schools. The PSS grew out of a proposal in 1988 to develop a private school data collection that would improve on the sporadic collection of private school data dating back to 1890 and improve on commercially available private school sampling frames. The PSS is currently designed to generate biennial data on the total number of private schools, teachers, and students, and to build a universe of private schools to serve as a sampling frame for the NCES sample surveys. The PSS was first collected by the U.S. Census Bureau in the 1989–90 school year, with data collections every two years since.

Target Population

The target population for the PSS is all schools located in the 50 states and the District of Columbia that are not supported primarily by public funds, provide instruction for one or more of grades kindergarten through 12 (or comparable ungraded levels), and have one or more teachers. Organizations or institutions that provide support for home schooling, but do not provide classroom instruction are not included.

Content

The 2005–2006 PSS instrument consisted of a single school questionnaire designed to be filled out by school administrators and is available on the PSS website at http://nces.ed.gov/surveys/pss. Data collected included enrollment by grade, enrollment by race/ethnicity and sex, number of high school graduates from the previous year, number of teachers, program emphasis, school religious orientation or affiliation, association membership, existence and type of kindergarten program, number of days in the school year, length of the school day, and whether the school has a library media center.

Top

Frame Creation

Because the PSS attempts to include every private school in the United States, a universe list of private schools meeting the PSS definition had to be created for the 2005–2006 PSS. Since 1983, NCES has used a dual-frame approach for building its private school universe (Beller 1984). The dual frame consists of a list frame and an area frame.

List Frame

The list-building component was the primary means for improving coverage of private schools. The basis for the 2005–2006 PSS list frame was the 31,287 private schools from the 2003–2004 PSS list frame and the certainty portion of the area frame. In order to provide coverage of private schools founded since 2003 and to improve coverage of private schools existing in 2003, NCES received and processed membership lists from 29 private school associations and religious denominations, and private school lists from the departments of education from each of the 50 states and the District of Columbia. Additional private schools that contacted NCES to be included in the PSS were added as list frame cases.

Beginning in 1995, the PSS private school definition was expanded to include schools for which kindergarten is the highest grade. In 2005, a separate list-building operation (Early Childhood Operation) was conducted to identify schools for which kindergarten was the highest grade (kindergarten terminal or k-terminal schools). Requests for lists of programs that might include a kindergarten were made to sources, other than state departments of education, in all 50 states and the District of Columbia, including state departments of health or recreation; state child care licensing agencies; and child care referral agencies. In 2005, some 27 of these lists were received, and 19 lists were processed (due to resource constraints not all received early childhood lists were processed in 2005).

Schools on private school association membership lists, the state lists, and the early childhood lists were compared to the base list, and any school that did not match a school on the base list was added to the universe list. Additionally, questionnaires were sent out to programs identified in the 2003–2004 PSS as prekindergarten only. This procedure was done in case any of these programs included at least a kindergarten in the 2005–06 school year. A total of 38,695 schools (unweighted) were included in the 2005–2006 list frame (table B-1).

Table B-1.  Weighted and unweighted number of interviews, noninterviews, and out-of-scope cases, and weighted and unweighted response rates, by frame: 2005–06
	List frame	Area frame	Total frame
Total	38,695	9,354	48,049
(unweighted)	(38,695)	(1,806)	(40,501)
Interviews	29,178	3,687	32,865
(unweighted)	(29,178)	(606)	(29,784)
Noninterviews	1,771	388	2,159
(unweighted)	(1,771)	(96)	(1,867)
Out-of-scope	7,746	5,279	13,025
(unweighted)	(7,746)	(1,104)	(8,850)
Response rate	94.3%	90.5%	93.8%
(unweighted)	(94.3)	(86.3)	(94.1)
NOTE: Weighted using the inverse of the probability of selection.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Private School Universe Survey (PSS), 2005–2006.

Area Frame

To identify private schools that may have been overlooked in the list-building component, a group of geographic areas was selected to be searched for private schools. The United States is divided by the Census Bureau into 2,062 primary sampling units (PSUs), each PSU consisting of a single county, independent city, or group of contiguous counties. The area frame consists of a sample of these 2,062 PSUs. The 2005–2006 PSS area frame was designed to produce approximately 50 percent overlap with the 2003–04 area frame in order to maintain the reliability of estimates of change at a reasonable level.

A total of 124 distinct PSUs (162 counties) were in the 2005–2006 PSS area frame sample. Within each of these PSUs, the Census Bureau attempted to find all eligible private schools. A block-by-block listing of all private schools in a sample of PSUs was not attempted. Rather, regional office field staff created the frame by using such sources as yellow pages, local Catholic dioceses, religious institutions, local education agencies, and local government offices. Once the area search lists were constructed, they were matched with the NCES private school universe list. Schools that did match the universe list were deleted from the area frame. A total of 1,806 schools (unweighted) were added to the universe from the area frame (table B-1).

Top

Data Collection

The 2005–2006 Private School Universe Survey utilized a mailout/mailback collection methodology. Most schools were also given the option to complete the questionnaire online.¹ Follow-up of non-responding schools was conducted first by computer-assisted telephone interviewing (CATI), and then by Census Bureau field representatives.

Of the 40,501 unweighted cases included in the 2005–2006 PSS, 8,850 cases were considered as out-of-scope (not eligible for the PSS). A total of 29,784 private schools completed a PSS interview, while 1,867 schools were noninterviews (refusals and nonrespondents) resulting in an unweighted response rate of 94.1 percent.²

Quality Control and Editing

Data from the interviews went through several edits; the edits consisted of:

• range checks to eliminate out-of-range entries;
• consistency edits to compare data in different fields for consistency;
• edits to verify that skip patterns on the questionnaire had been followed; and
• interview status recode (ISR), a program used prior to the weighting process to assign the final interview status to the record.

Imputation

Item Response Rates

The weighted item response rates for the variables used in this report are all greater than 85 percent. Values were imputed to items with missing data to compensate for item nonresponse.

First-Stage Imputation

In the first stage of imputation, values for missing items were imputed whenever possible by using information about the school from these sources:

• Other items on the 2005–2006 PSS—Based on entries from related items on the questionnaire, assumptions were made about how the respondent should have answered items with missing values. For example, if the respondent did not answer item 10a (length of school day for kindergarten, transitional kindergarten, transitional first grade) and item 5 indicated that the school did not have students enrolled in any of those grades, zero (school does not offer kindergarten) was imputed to item 10a.
• Data from the 2003–2004 PSS—Data from the 2003–2004 PSS were used to fill the items with missing values whenever possible. For example, if the type of school was not reported in item 12a and it had been reported on the 2003–2004 PSS, the 2003–2004 response was copied to item 12a.

In addition to filling items where values were missing, some inconsistencies between items were corrected by ratio adjustment. For records where the number of students in item 7 (students by race) did not equal the enrollment in item 6, excluding prekindergarten, the item 7 entries were adjusted to be consistent with item 6, i.e., each entry in item 7 was multiplied by the ratio of the student count in item 7 to the enrollment in item 6, excluding prekindergarten.

Second-Stage Imputation

Items still blank after first-stage imputation were imputed during the second-stage imputation. The second-stage imputation process used a hot-deck imputation methodology by extracting data from the record for a school (donor) similar to the nonrespondent. Variables that describe certain characteristics of the schools (e.g., religious affiliation, size, instructional level) were created and used to sort the records and to match incomplete records to those with complete data. Items 13 and 15 did not go through the donor imputation program. If, after the first stage, the item was still incomplete, the assumption was made for item 13 that the school was not in a private home, and for item 15 that the school did not belong to any associations. Items 14a and 14c (religious orientation and affiliation) did not go through the donor imputation program. Rather, if values were still missing after the first-stage imputation, the records were reviewed and imputed clerically.

Clerical Imputation

For a few items, there were cases where entries were clerically imputed. The data record, sample file record, and the questionnaire were reviewed and an entry consistent with the information from those sources was imputed. This procedure was used when (1) there was no suitable donor found, (2) the computer method produced an imputed entry that was unacceptable, and (3) the nature of the item required an actual review of the data rather than a computer-generated value.

Top

Weighting

The survey data from the area-frame component were weighted to reflect the sampling rates (probability of selection) of the PSUs. Survey data from both the list- and area-frame components were adjusted for school nonresponse. There were 29,784 interviews and 1,867 cases that were noninterviews. After weighting the area-frame component, these became 32,865 interviews and 2,159 noninterviews—the weighted response rate was 93.8 percent (table B-1).

Sampling Error

The standard error indicates how much variability there is in the population of possible estimates of a parameter for a given sample size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval.

Estimates of standard errors were computed using a technique known as half-sample replication. As with any replication method, half-sample replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean squared error of the replicate estimates around the full sample provides an estimate of the variance of the statistic (Wolter 1985). A computer program (SUDAAN), developed by Research Triangle Institute, was used to calculate the estimates of standard errors. For PSS, only the area frame contributes to the standard error. Since all list-frame cases are included in the PSS, the list frame component of the standard error is always zero (0). Standard errors for statistics in each table are presented in corresponding standard error tables.

Tests of Significance

The tests of significance used in this analysis are based on Student's t statistics. The t statistic between estimates from various subgroups presented in the tables can be computed by using the following formula:

where x₁ and x₂ are the estimates to be compared (e.g., the means of two groups) and SE₁ and SE₂ are their corresponding standard errors.

As the number of comparisons that are conducted at the same significance increases, it becomes more likely that at least one of the estimated differences becomes significant merely by chance, that is, will be erroneously identified as significantly different from zero. Even when there is no statistical difference between the estimates or percentages being compared, there is a 5 percent chance of getting a significant t value of 1.96 from sampling error alone. As the number of comparisons increases, the incidence of this type of error also increases.

Top

Nonsampling Error

Survey estimates are also subject to errors of reporting and errors made in the collection of the data. These errors, called nonsampling errors, can sometimes bias the data. Nonsampling errors may include such things as differences in the respondents' interpretation of the meaning of the questions, differences related to the particular time the survey was conducted, or errors in data preparation. While sampling theory can be used to estimate the sampling variability of an estimate, nonsampling errors usually require either an experiment conducted as part of the data collection procedure or use of data external to the study.

Undercoverage in the list and area frames is another possible source of nonsampling error. To reduce the possible bias caused by undercoverage in the list frame, the area frame was used to complement the list frame through the identification of schools missing from the list frame.

Publication Criteria for PSS Estimates

NCES criteria for the publication of an estimate are based on two category types: (1) surveys based on a sample from the frame or (2) surveys based on the entire frame (universe surveys). To publish an estimate for a sample survey, at least 30 cases must be used in developing the estimate; for a universe survey, at least three cases must be used in developing the estimate. PSS has pieces of both categories: (1) an area-frame sample of 124 PSUs which collects data on schools not on the list frame and (2) a complete census of schools belonging to the list frame. Since a significant component of a PSS estimate comes from the census of the list frame, the criterion of at least 30 cases seems unnecessarily high. However, since the area-frame component of an estimate can have a large sampling error, some indication of a large sampling error is appropriate. Based on these two concerns, estimates reported in a PSS table must be based on at least 15 schools. Some of the estimates in this report are footnoted with "Interpret data with caution: The coefficient of variation is larger than 25 percent." The coefficient of variation measures the precision of an estimate and is defined as the standard error of an estimate divided by the value of the estimate.

Top

---

¹ Amish schools on the list frame and all area-frame schools were not given the option to complete the questionnaire online.
² The response rate is calculated as follows: number of responding schools divided by the number of responding schools plus the number of nonresponding schools.