SASS Principal Follow-Up Survey (PFS)

4. SURVEY DESIGN

TARGET POPULATION

The target population for the 2008–09 PFS included principals in public, private, and BIE-funded schools, with students in any of grades K-12 or in comparable ungraded levels, which were in operation in school year 2007–08. All public, private, and BIE school principals who replied to a 2007–08 SASS principal questionnaire were included in the 2008–09 PFS sample.

The target population for the 2012–13 PFS included principals in public, public charter, and private schools, with students in any of grades K-12 or in comparable ungraded levels, which were in operation in school year 2011–12. All public and private school principals who replied to a 2011– 12 SASS principal questionnaire were included in the 2012– 13 PFS sample; therefore, any discussion of PFS methodology builds upon the preceding SASS methodology.

The 2016–17 PFS target population included principals in public and public charter schools with students in any of grades K-12 or in comparable ungraded levels and in operation in the 2015–16 school year. All public schools whose principal replied to the 2015–16 NTPS principal questionnaires were included in the PFS sample. For details on sampling at all levels of NTPS, see the Survey Documentation for the 2015–16 National Teacher and Principal Survey (Cox et al., forthcoming).

In SASS, the sampling frame for public schools and BIE schools was an adjusted version of the NCES Common Core of Data (CCD), and the sampling frame for private schools was a modified version of the NCES Private School Universe Survey (PSS). The sampling frame for the SASS principal questionnaire consisted of the principal of each sampled school in the SASS sample. The same sampling frame was used for NTPS, except that the 2015–16 NTPS did not include private schools.

SAMPLE DESIGN

All public (for SASS and NTPS) and private (for SASS only) school principals who replied to a SASS or NTPS principal questionnaire were included in the PFS sample. In the 2007–08 SASS, the principal of each sampled school was automatically selected. Altogether, 12,910 school principals were sampled. The 2008–09 PFS sample included all principals who completed SASS interviews in eligible schools. A total of approximately 9,480 schools were contacted for this survey; 2,700 eligible schools were not included because the principal did not respond to the Principal Questionnaire during the 2007–08 SASS; and 730 schools were deemed ineligible for 2007–08 SASS and not included in 2008–09 PFS.

In the 2011–12 SASS, the principal of each sampled school was also automatically selected. Altogether, 14,000 school principals were sampled (11,000 public and 3,000 private). The 2012–13 PFS sample included all principals who completed SASS interviews in eligible schools. A total of approximately 9,235 schools were included in the sample for this survey; 4,765 eligible schools were not included because the principal did not respond to the Principal Questionnaire during the 2011–12 SASS. However, only 9,230 schools were contacted for this survey; approximately 10 schools that completed a SASS interview were deemed ineligible from the 2011–12 SASS because their district refused participation after the SASS and were not contacted during the 2012–13 PFS.

The validation sample for the 2012–13 PFS consisted of 690 principals. The sample was divided between those principals in the same job (“stayers”) and those reported to be doing something else (“nonstayers”). The nonstayers were stratified by the major status category, whose values were as follows:

• 1 = Working as a principal at another school;
• 2 = Working in a K-12 school but not as a principal;
• 3 = Working in K-12 education but not at a school;
• 4 = Working outside of K-12 education; and
• 5 = Not working (including retired, deceased, on leave).

Each of these status categories was allocated an approximately equal sample. As responses were received, validation study cases were selected for sample on a flow basis. Of the 690 cases, approximately 190 cases were not attempted for validation because no home contact information was provided in SASS. There were an additional 50 cases that were not interviewed for the validation study because they refused or otherwise could not be contacted.

 

Top

Data Collection and Processing

The 2008–09 PFS, 2012–13 PFS, and 2016–17 PFS utilized
a primarily mail-based collection methodology, with telephone follow-up for nonrespondents. The 2012–13 PFS also utilized e-mail follow-up for nonrespondents. The U.S. Census Bureau was the data collection agent.

Reference dates. Data items refer to principal status at the time of questionnaire completion.

Data collection. In the spring of the year of the survey administration, an introductory letter and a Principal Status Form were mailed to sampled schools in the previous year’s SASS or NTPS. On this form, an eligible respondent including any school employee knowledgeable about the current status of last year’s principal was asked to report (1) the current occupational status and (2) the name of last year’s principal. Approximately two weeks after initial mailings, a reminder letter and a second copy of the Principal Status Form were mailed to all sampled schools to remind them to complete and return the questionnaire. Following these contacts, telephone nonresponse follow-up of schools was conducted. At all stages of data collection, procedures were in place to remail questionnaires by request.

Because a total of 830 schools were located in districts identified as special procedure districts in the 2007–08 SASS, which required district approval prior to allowing their schools to participate in surveys, an introductory letter and a sample copy of the Principal Status Form was mailed in February 2009 to 18 of these special districts. Data collection for the 2008–09 PFS began in March 2009 with the mailing of an introductory letter and the Principal Status Form to sampled schools in the 2007–08 SASS. In March 2009, a reminder letter and a second copy of the Principal Status Form were also mailed to all sampled schools. Telephone nonresponse follow-up of schools was conducted from March through April 2009. Although all follow-up was completed prior to May, mailed questionnaires were accepted through early June 2009.

Data collection for the 2012–13 PFS began in March 2013 with the mailing of an introductory letter and the Principal Status Form to sampled schools in the 2011–12 SASS. In March 2013, a reminder letter and a second copy of the Principal Status Form were also mailed to any outstanding sampled schools. Telephone nonresponse follow-up of schools was conducted in April 2013. Although all follow- up to the schools was completed prior to May 2013, mailed questionnaires were accepted through early June 2013. In May 2013, any outstanding principals from the initial mailing to the schools in March 2013 were mailed an introductory letter and the Principal Status Form directly to the principal’s home. Telephone nonresponse follow-up directly to the principal’s home was conducted from May to June 2013.

A validation study was conducted with the 2012–13 PFS. As completed school-level questionnaires were received, principals were sampled to be a part of the validation study. In April 2013, an introductory letter and a Principal Status Form were mailed to the sampled principal’s home for the validation study. These packages were mailed weekly until the beginning of May 2013, as eligible principals were sampled. An email reminder was sent to nonresponding validation study principals. Telephone nonresponse follow- up directly to the principal’s home for those selected for the validation study was conducted from April to June 2013. Mailed questionnaires were accepted through early June 2013.

Data collection for the 2016–17 PFS began in early March 2017. An introductory letter and the Principal Status Form (i.e., the survey questionnaire) were mailed to the sampled schools. The letter introduced the survey and asked the school to complete and mail the questionnaire in the return envelope. The letter was addressed to the principal, or if the principal’s name had not been provided during the 2015–16 NTPS data collection, it was addressed to the “School Principal/Administrator.” Eligible respondents included any school employee with knowledge of the 2015–16 principal’s employment status.

To ensure the validity of responses collected from schools, the PFS Home Contact Operations was conducted. Once complete Principal Status Forms were returned by schools, principals determined to be “non-stayers” based on school responses were assigned to a validation study group. Between early April and mid-May, the appropriate public Principal Status Form was mailed to the homes of these subsampled principals and the data collected was used to validate the form returned by the school. For additional information on the PFS validation study, see Survey Documentation for the 2016–17 Principal Follow-up Survey (Khouri, forthcoming).

Nonresponse follow-up followed each stage of data collection. In mid-March 2017, a reminder letter and second copy of the Principal Status Form were mailed to any outstanding sampled schools. In early May 2017, any outstanding principals from schools that did not respond to the initial mailing in March were mailed an introductory letter and the Principal Status Form, sent directly to the principal’s home. In instances where a school or principal had provided a principal’s e-mail address, initial contacts and reminders were sent via e-mail. Telephone nonresponse follow-up was conducted from mid- to late April for schools and from late May through early June for principals. Mailed questionnaires were accepted through the middle of July 2017.

Editing. The PFS data were converted from paper to electronic format using manual data keying and imaging technology. Responses were recorded; questionnaire data were captured; output files were reformatted in order to facilitate the remaining data processing and cleaning and were sent for data review. During data review, discrepancies were uncovered, researched, and resolved.

The 2008–09 PFS and the 2012–13 PFS classified cases into two categories either as completed interviews or as noninterviews. The 2012–13 PFS further classified out-of- scope cases into an additional category.

PFS data were subjected to a range of data quality reviews before release. General data quality checks for the PFS involved an examination of the individual responses, patterns of response, and summary statistics for variables to ensure internal consistency.

Top

Estimation Methods

Estimates from PFS sample data were produced using weighting and imputation procedures.

Weighting. The general purpose of weighting is to scale up the sample estimates to represent the target survey population. Prior to the 2016–17 PFS, all responding SASS principals were included in the PFS, and since the PFS response rates were high, the SASS weights can be used to represent the target sample population. All units, respondents and nonrespondents to the PFS, were included in the file and no adjustments were made for nonresponse. Therefore, new weights were not calculated for the PFS, and data users should employ the SASS principal weights when analyzing the data files. For more information on SASS weighting, please refer to the Documentation for the 2011– 12 Schools and Staffing Survey (Cox et al., 2016).

For the 2016–17 PFS, new weights were not calculated because the PFS instrument was sent to all schools whose principals were interviewed in NTPS. Therefore the 2015– 16 NTPS public school principal weights should be used when analyzing the Public School Principal Status Data File, respectively. This weight, to be used on all principal status data files, is PFNLWGT. For more information on NTPS weighting, please refer to the Survey Documentation for the 2015–16 National Teacher and Principal Survey (Cox et al. forthcoming).

Imputation. SASS and NTPS are fully imputed datasets. No imputation is performed in PFS. The principal status created variables have missing values, which only occur when a case is considered a noninterview in the PFS. These values are represented in the files as a -8.

FUTURE PLANS

The next administration of PFS is planned for the 2020–21 school year as a follow-up to NTPS 2019–20.

Top