Survey of Earned Doctorates (SED)

4. SURVEY DESIGN

TARGET POPULATION

The population consists of all individuals that are awarded research doctorates from accredited colleges and universities in the United States and Puerto Rico between July 1 and June 30. For the 2011 academic year, which covered the period of July 1, 2010, through June 30, 2011, the total universe consisted of 49,010 persons in 412 institutions that conferred research doctorates.

SAMPLE DESIGN

The SED is a census of all individuals receiving a research doctorate from a U.S. institution in a given academic year. To establish this universe of individuals, the universe of research-doctorate-granting institutions is determined. Institutions are eligible for participation in the SED if they (1) grant research doctoral degrees and (2) are accredited by a nationally recognized accrediting agency. Determination of eligibility begins with a review of the most recent release of the U.S. Department of Education‘s IPEDS’ Institutional Characteristics and Completions datasets. This procedure, known as the “SED universe review,” identifies institutions that may be added to the SED universe. The SED universe review also helps in flagging institutions that might be removed from the SED universe for any of a variety of reasons (e.g., abolition of its research doctoral program, merging with another institution, etc.). Once institutions have been identified for further review, SED staff members consult institution web sites and other sources to determine preliminary eligibility.

If the institution’s doctoral degrees appear to meet the SED’s definition of “research doctorate,” the survey contractor’s task leader for institution contacting sends the institution’s graduate dean a brief questionnaire to confirm eligibility for inclusion in the SED. If eligibility is determined, the institution’s participation in the survey is requested.

Data Collection and Processing

The data collection and editing process spans a 18-month period ending 6 months after the last possible graduation date (i.e., June 30); the process is a collaborative effort among institutions, the survey contractor, and the doctorate recipients.

The institutions assist the survey contractor by identifying new doctorate recipients, distributing questionnaires, and providing names and mailing and email addresses for all graduates who did not return completed questionnaires. Most institutions distribute the SED with other final paperwork.

Four months before the data collection field period closes, an accelerated mail cycle is implemented in which non-respondents receive three letters, each timed one week apart. A follow–up phone survey is also conducted targeting non-respondents who were unresponsive for the full cycle of prompts and follow–ups The data collection contractor additionally locates and follows up with graduates regarding surveys not returned and regarding information missing from the surveys and assists institutions with increasing their response rates if needed.

Reference Dates. Data are collected for an academic year, which includes all graduations from July 1 of one year through June 30 of the following year.

Top

Data Collection. The doctorate institution is responsible for administering the surveys to research doctoral candidates and, for the hard-copy version of the survey, collecting the completed questionnaires for mailback to the survey contractor. The doctorate recipients themselves complete the surveys.

There are three modes of data collection used in the SED: self–administered paper surveys, Web–based surveys, and computer–assisted telephone interviews (CATI). Paper surveys are mailed to institution coordinators in the graduate schools who distribute the surveys to students receiving research doctorates. The institution coordinators collect the completed surveys and return them to the NSF survey contractor for editing/processing. Since 2001, a Web–based SED option has been available. In addition to or, at some universities, instead of providing paper surveys to students when they applied for graduation, institution coordinators distribute a link to the SED survey registration website. Upon registering at the SED survey website, students receive PIN and password information via e–mail, as well as the URL of the Web–based SED. The NSF survey contractor uses both the paper and the Web–based SED to conduct follow–up interviews with nonrespondents. The proportion of completed surveys from respondents using the Web–based SED has increased each year since 2001, and in 2011 reached 39%. Starting in 2005, CATI was used to administer an abbreviated questionnaire to nonrespondents. Approximately 1%–2% of SED respondents use the CATI–based SED each year. The NSF survey contractor also mails to individual respondents and institutions a paper survey when critical SED questionnaire items are missing.

During SED data collection, it is essential to collect a complete college education history. To code these data, the SED uses the IPEDS coding frame for the U.S. institutions where doctorate recipients earned their baccalaureate and/or master's degrees. Because one–third of doctorate recipients from U.S. universities are citizens of foreign countries, a coding manual for foreign institutions of higher education was developed by the U.S. Department of Education (it is available at http://www.nsf.gov/statistics/mapping/).

Data Processing. To retrieve missing information on critical data items, missing information letters (MILs) are sent to individuals and missing information rosters (MIRs) are sent to institutions in efforts to retrieve data on the following: year of birth, sex, citizenship status, country of citizenship, race/ethnicity, baccalaureate institution, baccalaureate year, and postdoctoral location. The importance of this follow–up is emphasized in several ways: by sending selected MIR mailings by Federal Express, by sending MILs via email as well as regular mail, and by allowing institutions to provide data by email, phone or fax.

Since academic year 2003, data processing has included several procedures designed to reduce the burden of retrieving missing information on institutions and individuals. For instance, the survey contractor incorporates data abstracted from the school materials into the receipt process for key fields such as doctoral field of study and doctorate type. As well, other items such as master’s institution, master’s date, master’s field of study, bachelor’s institution, bachelor’s date, and bachelor’s field of study can be abstracted from the retrieval materials.

Editing. While many sample surveys use statistical routines to impute missing data, the SED relies primarily on follow–up activities and data cleaning to increase item response levels. Some improvements are also realized from machine editing and cleaning, which decrease nonresponse by imposing inter-item consistency and logical agreement between related questions. In addition, responses to particular items can allow missing items to be logically assigned. For example, the missing response to a filter question can often be inferred if dependent questions have been answered.

Top

Estimation Methods

No weighting is performed since the SED is a census.

Future Plans

Additional changes to the SED are under consideration, both to capture new data relevant to current issues in graduate education and to collect better data through existing questions.

Top