Integrated Postsecondary Education Data System (IPEDS)

5. DATA QUALITY AND COMPARABILITY

Data element definitions have been formulated and tested to be relevant to all providers of postsecondary education and consistent among components of the system. A set of data elements has been established to identify characteristics common to all providers of postsecondary education, and specific data elements have been established to define unique characteristics of different types of providers. Interrelationships among various components of IPEDS have been formed to avoid duplicative reporting and to enhance the policy relevance and analytic potential of the data. Through the use of “clarifying” questions that ask what was or was not included in a reported count or total or the use of context notes that supplement the web collection, it is possible to address problems in making interstate and inter-institutional comparisons. Finally, specialized, but compatible, reporting formats have been developed for the different sectors of postsecondary education providers. This design feature accommodates the varied operating characteristics, program offerings, and reporting capabilities that differentiate postsecondary institutional sectors, while yielding comparable statistics for some common parameters of all sectors.

Sampling Error

Only the data collected prior to 1993 from a sample of private less-than-2-year institutions are subject to sampling error. With this one exception, the HEGIS and the IPEDS programs include the universe of applicable postsecondary institutions.

Nonsampling Error

The IPEDS data are subject to such nonsampling errors as errors of design, reporting, processing, nonresponse, and imputation. To the extent possible, these errors are kept to a minimum by methods built into the survey procedures.

The sources of nonsampling error in the IPEDS data vary with the survey instrument. In the Fall Enrollment component, the major sources of nonsampling error are classification problems, the unavailability of needed data, misinterpretation of definitions, and operational errors. Possible sources of nonsampling error in the Finance component include nonresponse, imputation, and misclassification. The primary sources of nonsampling error in the Completions component are differences between the NCES program taxonomy and taxonomies used by colleges, classification of double majors and double degrees, operational problems, and survey timing. A major source of nonsampling error in the Graduation Rates components is the correct identification of cohort students (full-time, first-time, degree/certificate-seeking undergraduates); for Human Resources, difficulties in classifying employees by primary occupation; for 12-month Enrollment, definitional difficulties with calculating instructional activity. For Student Financial Aid, institutions often must merge enrollment and financial aid databases, and face difficulties in placing students in the various groups for which data are collected.

Top

Coverage error. Coverage error in IPEDS is believed to be minimal. For institutions that are eligible for Title IV federal financial aid programs, coverage is almost 100 percent. Schools targeted as “possible adds” are identified from many sources, including a review of the PEPS file from OPE, a universe review done by state coordinators, and the institutions themselves.

Nonresponse error. Since 1993, all institutions entering into PPAs with the U.S. Department of Education are required by law to complete the IPEDS package of components. Therefore, overall unit and item response rates are quite high for all components for these institutions. Data collection procedures, including extensive email and telephone follow-up, also contribute to the high response rates. Imputation is performed to adjust for both partial and total nonresponse to a survey. Because response rates are so high, error due to imputation is considered small.

Unit nonresponse. Because Title IV institutions are the primary focus of IPEDS and they are required to respond, overall response rates for Title IV institutions and administrative units are high. For example, the response rates in the Spring 2018 IPEDS collection were high: greater than 99 percent for each survey component. Since the implementation of the web collection, Title IV institutional response rates for the various IPEDS surveys have ranged from about 89 percent to over 99 percent.

By sector, the response rates are highest for public 4 year or higher institutions and lowest for private for–profit institutions, especially less–than–2–year institutions. The 1994 Academic Libraries Survey and the FY 95 Finance public–use data files are limited to IHEs because the response rates for postsecondary institutions not accredited at the collegiate level were quite low (specifically: 74 percent in the Finance collection and less than 50 percent in the Academic Libraries Survey).

Item nonresponse. Most participating institutions provide complete responses for all items. Telephone and email follow-up are used to obtain critical missing items.

Measurement Error. NCES strives to minimize measurement error in the IPEDS data by using various quality control and editing procedures. New questionnaire forms or items are field tested and/or reviewed by experts prior to use. To minimize reporting errors in the Finance component, NCES uses national standards for reporting finance statistics. Wherever possible, definitions and formats in the Finance component are consistent with those in the following publications: College and University Business Administration; Administrative Services, Financial Accounting and Reporting Manual for Higher Education; Audits of Colleges and Universities; and HEGIS Financial Reporting Guide.

The classification of students appears to be the main source of error in the Enrollment (especially the fall section that is now a separate component) component. Institutions have had problems in correctly classifying first-time freshmen, other first-time students, and unclassified students for both full-time and part-time categories. These problems occur most often at 2-year institutions (both public and private) and private 4-year institutions. In the 1977–78 HEGIS validation studies, misclassification led to an estimated overcount of 11,000 full-time students and an undercount of 19,000 part-time students. Although the ratio of error to the grand total was quite small (less than 1 percent), the percentage of errors was as high as 5 percent at student detail levels and even higher at certain aggregation levels.

Top

Data Comparability

The definitions and instructions for compiling the IPEDS data have been designed to minimize comparability problems. However, survey changes necessarily occur over the years, resulting in some issues of comparability. Also, postsecondary education institutions vary widely, and hence, comparisons of data provided by individual institutions may be misleading. Specific issues related to the comparability of the IPEDS data are described below.

Classification of institutions. Since 1996, the subset of IPEDS institutions that are eligible to participate in Title IV federal financial student aid has been validated by matching the IPEDS universe with the PEPS file maintained by OPE. Prior to 1996, institutions were identified as aid-eligible from the list of IHEs and responses to the Institutional Characteristics component.

Fields of study. In analyzing Completions data by field of study, users must remember that the data are reported at the institution level and represent programs, but not schools, colleges, or divisions within institutions. For example, some institutions might have a few computer and information science programs organized and taught within a business school. However, for IPEDS reporting purposes, the degrees are classified and counted within the computer and information science discipline.

Reporting periods. The IPEDS survey is separated into 12 components, which correspond to three seasonal reporting periods. The Institutional Characteristics, Completions, and 12-month Enrollment surveys are administered in the fall. The Student Financial Aid, Graduation Rates, 200% Graduation Rates, Outcome Measures, and Admissions components are collected in the winter. The Fall Enrollment, Finance, Academic Libraries, and Human Resources components are administered in the spring.

Top

Survey changes. Over the years, the IPEDS survey forms have undergone revisions that may have an impact on data comparability. Users should consider the following:

Revisions to the CIP were made in 1970, 1980, 1985, 1990, 2000, and 2010.

Racial/ethnic data for Fall Enrollment have been collected annually since 1990 (biennially, in even–numbered years, before then). Additional items were included on students enrolled in branch campuses in foreign countries, students enrolled exclusively in remedial courses, and students enrolled exclusively at extension divisions; however, these items were discontinued in 2000. Prior to 1996, data were also collected in even–numbered years from 4–year institutions for the fields of Veterinary Medicine and Architecture and Related Programs.

From 1990 to 1994, racial/ethnic data (by gender and degree/award level) were collected at the 2–digit CIP level on the Completions component. In 1995, there was a major restructuring of the component to collect race/ethnicity at the 6–digit CIP level and to add additional questions to collect numbers of completers with double majors and numbers of degrees granted at branch campuses in foreign countries. The additional questions were dropped in 2000–01, but a matrix to collect completions data on multiple majors was instituted for optional use in 2001–02 and became mandatory in 2002–03.

In fall 1995, the salary class intervals were revised for the Fall Staff component. Salary class intervals were revised again in 2001.

Over the years, the various versions of the Finance form have changed. Prior to 1997, the survey forms for public and private institutions were basically the same except that the public institution form contained three additional sections, with data from questions pertaining to state and local government financial entities used by the U.S. Bureau of the Census.

The Finance form for private institutions was revised in 1997 to make it easier for respondents to report their financial data according to new standards issued by the Financial Accounting Standards Board (FASB). In an attempt to address the reporting issues of proprietary institutions, the for-profit form was revised in 1999 to reflect the financial statements of these institutions. Due to new accounting standards issued by the Governmental Accounting Standards Board (GASB), beginning optionally in 2002, with a 2–year phase–in period, public GASB reporting institutions moved from fund–based reporting to whole–entity reporting that is more similar to the private FASB–reporting institutions.

With the web–based data collection in 2000–01, the number of data items requested from institutions was greatly reduced.

Salary outlays, total number of instructional staff, and tenure status were collected for full–time staff on less than 9–month contract schedules through 1999–2000; currently only academic rank and gender are collected for these other contract schedules. Faculty status was not collected between 2001–02 and 2004–05, and was reinstated for degree-granting institutions in 2005–06. The reporting of data by faculty status was optional for 2005–06, but was required beginning in 2006–07. Beginning with the 2004–05 data collection, only degree–granting institutions have been required to complete the SA section of the HR component.

Prior to 2000–01, the Graduation Rates component collected additional data on students’ length of time to complete; the number of students still persisting; and the number of students receiving athletically related student aid and their length of time to complete. The sections of the component collecting data on students receiving athletically related student aid were discontinued with the 2007–08 data collection.

Prior to 2001, the Fall Staff component requested the number of persons donating (contributing) services or contracted for by the institution.

As of the 2004–05 collection, IPEDS has limited the collection of data on employees in medical schools to institutions with an M.D. and/or D.O. program. In previous collections, all 4–year institutions were given the opportunity to report employees in medical schools. However, some institutions that did not have a medical school erroneously reported employees in this section of the Employees by Assigned Position section. This change may cause some discrepancies in comparisons of the IPEDS medical school data.

Accreditation information was collected in the IC component until 2006–07, when the Office of Postsecondary Education opened its database and searchable web tool of accredited institutions (https://ope.ed.gov‌/accreditation/), collecting data from the accreditation agencies.

A 2–year phase–in period began with the 2008–09 collection to implement additional changes to better align the finance reporting of public and private institutions. Since the 2010–11 collection, all public and nonprofit institutions have used the newly–aligned form. Starting in 2014–15, private–for–profits started to report Finance data more like the public and private non–for–profit institutions.

The 2008–09 data collection was the start of a 3–year phase–in to the reporting of the new, 1997 federal race and ethnicity categories. The new categories allow students and staff to identify themselves using two or more race categories. The transition to the new race and ethnicity categories was required for the 2010–11 data collection of the Fall Enrollment and Human Resources components, and was required for all other IPEDS components as of the 2011–12 data collection.

The 2008–09 data collection was the start of a 2–year phase–in of the restructuring of the postbaccalaureate degree categories. As of the 2010–11 data collection, the first–professional degree and certificate categories were eliminated, and the doctor’s degree category was expanded to three categories: research/scholarship, professional practice, and other. These changes reflect changes in graduate education over the years, and make it easier to distinguish research–focused doctor’s degrees from professionally focused doctor’s degrees.

In 2009–10, forms used to collect GR data for less than 4–year institutions were modified to include reporting of completers within 100 percent of normal time in addition to 150 percent of normal time. This change aligned forms for the less than 4–year institutions with the 4-year institutions’ forms.

For the 2009–10 data collection, additional changes to the SFA component were implemented due to the 2008 Higher Education Opportunity Act (HEOA) and for clarification, including the collection of average aid amounts for sub-groups of the full–time, first–time degree/certificate–seeking undergraduate population, to be used in the calculation of average institutional net price and average institutional net price by income category information for display on the College Navigator website (https://nces.ed.gov‌/collegenavigator/).
In 2014–15, the SFA component began collecting data on undergraduate and graduate students receiving military service members and veterans benefits.

Top

History of Classification of Instructional Programs. The purpose of the Classification of Instructional Programs (CIP) is to provide a taxonomic scheme that supports the accurate tracking, assessment, and reporting of fields of study and program-completions activity. NCES has utilized a number of versions of CIP throughout the life of IPEDS, as well as its predecessor, HEGIS.

In 1970, NCES published A Taxonomy of Instructional Programs in Higher Education that was to be used beginning with the HEGIS surveys of 1971–72. This taxonomy was divided into two main sections: one dealt with conventional academic subdivisions of knowledge and training; the other dealt with technologies and occupational specialties related to curricula leading to associate’s degrees and other awards below the baccalaureate. Both sections used 4–digit numerical codes to represent the fields.

In 1981, NCES published A Classification of Instructional Programs. In addition to new programs that evolved or gained new significance since 1970, there were weaknesses in the way instructional programs were classified and disaggregated. The new CIP instituted the current 6–digit code, which allowed obtaining data by 2–digit or 4–digit groups of fields more easily than the older scheme. The new CIP also included program definitions or descriptions that the 1970 version lacked, as well as other improvements.

In 1985, another revision to the CIP was released, although this was more of an update to the 1980 CIP than a radical change. There were 116 fields deleted, either due to duplication or because programs no longer existed to the degree needed for national reporting. Forty fields were added based on write-in entries on surveys returned. In addition, there were a few revisions of codes or names of fields. This CIP was used during the final years of HEGIS and continued into IPEDS.

A more extensive revision of CIP was released in 1990, which included programs at the secondary and adult education levels. Within the postsecondary level, there were several major restructures. Fields previously included in Business and Management (06) and Business (Administrative Support) (07) were integrated into a new Business Management and Administrative Support (52). Similarly, fields previously in Allied Health (17) and Health Sciences (18) were integrated into Health Professions and Related Sciences (51). Again there were deletions and additions, although many were actually combining two former fields into one, or vice versa. The 1990 CIP was first used in IPEDS in 1991–92.

A further revision resulted in publishing Classification of Instructional Programs: 2000 Edition in 2002. This CIP was adopted as the standard field of study taxonomy by Statistics Canada, based on the comprehensiveness and detail of the CIP and the potential for enhanced comparability with U.S. education data. Again, there were several major reorganizations. Fields previously reported in Agricultural Sciences (02) were divided between Agriculture, Agriculture Operations and Related Sciences (01) and Biological and Biomedical Sciences (26). Fields previously reported in Sales and Marketing Operations/Marketing and Distribution (08) were incorporated into Business, Management, Marketing, and Related Services (52). History became a separate 2–digit CIP (54) moved from Social Sciences and History (45). In addition, there were a large number of new fields added. The CIP–2000 was first used in IPEDS in 2002–03.

The web–based version of CIP was implemented during 2008–09, and is accessible online at https://nces.ed.gov/ipeds/cipcode/. The web–based CIP incorporates tools for browsing, searching, and crosswalking. Several additional changes were implemented in conjunction with this online version: 50 new 4–digit codes and 300 new 6–digit codes were added, and several series were reorganized (English Language and Literature/Letters (23), Psychology (42), Nursing (51.16), and Residency Programs (60)). One series was also deleted (Technology Education/Industrial Arts (21)), and several examples of instructional programs were added to assist users in selecting the appropriate field. These revisions have been utilized in IPEDS since 2010–11.

Top

Comparisons with HEGIS. Caution must be exercised in making cross–year comparisons of institutional data collected in IPEDS with data collected in HEGIS. IPEDS surveys request separate reporting by all institutions and their branches as long as each entity offers at least one complete program of study. Under HEGIS, only separately accredited branches of an institution were surveyed as separate entities; branches that were not separately accredited were combined with the appropriate entity for the purposes of data collection and reporting. Therefore, an institution may have several entities in IPEDS, where only one existed in HEGIS.

Comparison with the Survey of Earned Doctorates. Like the IPEDS Completions survey, the Survey of Earned Doctorates (SED; for more information, see the SED chapter) also collects data on doctoral degrees, but the information is provided by doctorate recipients rather than by institutions. The number of doctorates reported in the Completions component is slightly higher than in SED. This difference is largely attributable to the inclusion of nonresearch doctorates (primarily in theology and education) in the Completions component. The discrepancies in counts have been generally consistent since 1960, with ratios of the IPEDS-to-SED counts ranging from 1.01 to 1.06. Differences in the number of doctorates within a given field may be greater than the overall difference because a respondent to SED may classify his or her specialty differently from the way in which the institution reports the field in the Completions survey.

Top