PROPOSAL SUMMARY: NCES should create new derived and calculated variables based on relevant IPEDS data to address important policy issues. These new variables should be incorporated into the Peer Analysis System and other IPEDS tools where appropriate. Any derived data should be clean, reliable, and well defined. Specific derived variables are proposed.
The IPEDS Technical Review Panel (TRP) on Derived and Calculated Variables met on February 4-5, 2004. In attendance were 49 individuals, representing federal government, state government, system offices, institutions, national higher education associations, and contractors. The panel considered proposals to create new derived and calculated variables for IPEDS.Background:
As part of its effort to promote more effective and efficient use of the IPEDS national datasets, NCES is interested in creating derived and calculated variables. The availability of new derived data will make it quicker for users of the Peer Analysis System to get the data they need without having to make cumbersome calculations. Two types of derived data are of interest, including (1) subtotals of existing data; and (2) calculations of new data based on two or more existing variables. It is expected that these derived variables will be useful in addressing various current policy questions.
This effort to create IPEDS derived and calculated variables should be considered in the context of other related efforts: (1) the NPEC Data Feedback project; (2) the proposed calculation of instructional costs per student FTE; (3) improvements to the navigation system for the Peer Analysis System (PAS); and (4) creation of data for the new IPEDS Data Analysis System (DAS) and IPEDS Executive Peer Tool (ExPT).
The project began with comprehensive review of the higher education literature. A spreadsheet was prepared with 304 possible derived variables, coded by subject and topic. The data element dictionaries and IPEDS survey forms for the past several years were analyzed to see whether data from this list could be produced with IPEDS.
A revised draft was sent to selected members of the institutional research and association community. Per OMB regulations, sections of the list of derived variables were sent to no more than 9 persons for review. A total of 25 people provided feedback to the section they were given and a number of important issues were raised. The working list of potential variables was revised accordingly and then shared with IPEDS staff, who reviewed the list and provided their input.
As part of its meeting in February, the TRP broke into six subgroups representing the IPEDS survey topics. Subgroups discussed (1) the Human Resource (HR)-related surveys, including Salaries, Fall Staff, and Employees by Assigned Position; (2) the Graduation Rates survey (GRS); (3) the Finance survey (F); (4) the Enrollment survey (EF); (5) the Completions survey (C); and (6) the Institutional Characteristics (IC) and Student Financial Aid (SFA) surveys. Several of the groups continued their discussions for a week after the TRP meeting and the group chairs each prepared a revised list of suggestions and summarized the group's feedback.TRP Recommendations:
The Finance group recommended that two general caveats be documented, including that: (1) any measure of institutional expenses or revenues per FTE student will overestimate values to the extent that the institution offers non-credit instruction; and (2) comparisons of public and private institution expenses and comparisons across institutions in the public sector should be discouraged.
There were four new measures the Finance group considered. The first, a measure of tuition discounting, was included in the set of suggestions. The final three (investment rate of return, research productivity, and cost to produce a degree) were all rejected for various reasons. While the TRP explored various enrollment/cost ratios, it was unable to gain consensus.
The calculation of student FTE was discussed extensively. The TRP suggested that the methodology, which was proposed as part of the separate work on instructional expenditures per student FTE, be adopted, except for the calculation of first-professional student FTE. FTE for first-professional students should be calculated using data from the fall enrollment component, by adding the number of full-time students to one-third of the number of part-time students. While this is based on fall data and the undergraduate and graduate FTE calculations are based on 12-month activity, it is expected that this estimate adequately reflects the pattern of first-professional enrollment for the purposes of estimated FTE.
The group on HR surveys focused extensively on the need for subtotals and totals of qualifying variables. The group also discussed the possible use of service months to calculate FTE for faculty; however these data are not currently collected and will not be included IPEDS. The TRP did reach consensus about a proxy derived variable that is relatively simple to construct. Total FTE faculty should be equal to the number of full-time faculty plus the number of part-time faculty divided by three.
The group concerned with the graduation rates survey contributed several variables, which are listed below. The student financial aid and institutional characteristics group developed an improved method for calculating the average price of attendance.
Issues of imputation and perturbation were discussed. NCES staff explained that cell suppression is no longer an issue because of the new perturbation procedures. Data are imputed for both item nonresponse and total nonresponse in order to provide national estimates. The TRP members agreed that imputed data should be used in derived and calculated variables because they represent the best possible estimate available.Proposal:
The TRP suggestions can be classified as two basic types: (1) addition of new totals and subtotals; and (2) creation of new derived and calculated variables. It was suggested that subtotals be created for qualifying variables and data. The PAS will be redesigned to provide subtotals whenever possible. The TRP agreed that new variables should not be created in order to express percentages of totals (e.g., percent of total students that are Hispanic). Rather, the report templates, which are provided in the PAS, should be expanded to generate these percentages for the user.
A list of proposed new derived and calculated variables follows:
NCES is concerned about how the calculation of new derived and calculated variables will impact the use of institutional data to inform policy. We encourage interested parties to send any comments or concerns about this proposal to John Milam at firstname.lastname@example.org by December 1, 2004. Please include the following in the email subject line: Derived Variables TRP - Proposal comments. Copies of your email should also be sent to Elise Miller.