Statistical Analysis Report:

Student Outcomes Information for Policy-Making: Final Report of the National Postsecondary Education Cooperative Working Group on Student Outcomes From a Policy Perspective

April 1997

Student Outcomes Information for Policy-Making

In this ‘age of accountability,' administrators and others have been especially concerned about educational outcomes and their measurement. Postsecondary institutions . . . are also being called on to provide factual evidence that they and their programs are providing the benefits that were intended, and that these outcomes are being produced in a cost-effective manner (Lenning, 1977, p. ix).

Lenning's description of some of the challenges facing postsecondary education in the U.S. may be nearly two decades old, but some things are slow to change. Currently, many members of Congress, governors, state legislators, top business executives, and others consider the information currently available on Americas postsecondary institutions to be inadequate for evaluating the quality of the education being delivered and for making decisions on policy or the expenditure of public funds (e.g., Romer, 1995). A number of forces have combined to increase the pressure on administrators and faculty members in public institutions to demonstrate the educational effectiveness of their institutions. Three of the most pressing include:

Increased competition for public funds from other public services (e.g., highways, prisons, social services) even as the resource base in most states erodes;
State appropriations that fail to keep pace with inflation; and
Legislative demands for accountability and increased productivity.

While public institutions may feel these particular pressures more keenly than private institutions, other, equally-threatening sources of pressure make no distinction between public and private institutions in Americas postsecondary system. ("Postsecondary education," as used here, refers to formal instruction beyond that provided in secondary school. It may be delivered in traditional colleges or universities, proprietary schools, tribal colleges, the military, corporate or industrial education programs, or other, similar settings. It does not include self-directed learning, such as occasional programs that may be delivered by libraries or museums, nor knowledge/skill development acquired through other informal, unstructured means.) Postsecondary educational institutions in the independent sector share with their public counterparts susceptibility to such pressures as:

The spiraling costs of attendance and operation;
Accrediting agency requirements to assess student learning outcomes and to incorporate the findings in operational and programmatic planning and decision- making;
Corporate and government leaders alarm about Americas economic competitiveness in a global market;
Writers both inside the academy (e.g., Anderson, 1992; Bloom, 1987; Huber, 1992; Smith, 1990) and out (e.g., Sykes, 1988) who have attacked college and universities reward structures and current directions;
News stories of financial mismanagement (and in some cases fraud) or educational malfeasance at both prestigious and obscure institutions;
Colleges and universities reluctance to engage in outcomes assessment and slowness in responding to external requests for outcomes information, which have fueled legislative and public frustration and impatience;
Graduates who cannot find postbaccalaureate employment despite a healthy economy; and
Employers increasing dissatisfaction with the skills their new employees bring to the workplace from their baccalaureate programs.

Proprietary degree-granting institutions (perhaps because their programs are both shorter and more specifically targeted at employment than are those of not-for-profit institutions) appear to be less open to criticism relating to employer dissatisfaction, program completion, and employment placement rates than are institutions in other postsecondary sectors (e.g., Accrediting Commission of Career Schools and Colleges of Technology, undated). Their achievements notwithstanding, however, proprietary institutions confront many of the same educational "quality" issues facing their not-for-profit counterparts.

The National Postsecondary Education Cooperative (NPEC), with the support of the U.S. Department of Educations , was formed to develop ways to improve the utility of information for policy-making at all levels and in all sectors of the postsecondary education enterprise. NPEC is a compact of data users and providers with representatives from a wide array of individual campuses, state higher education systems, and postsecondary education organizations, agencies, and accrediting bodies. One of NPECs first seven projects involves identification of student outcomes that could inform policy-making at the campus, state, and national levels and in all postsecondary sectors. A variety of institutional performance indicators are, of course, already available (e.g., admissions, registration, and financial aid systems at the institutional and state levels, and IPEDS at the national level). While such data are often useful for addressing some policy issues, the information they yield tends to be restricted to students precollege characteristics and various "production counts" (e.g., students enrolled, credit-hours produced, degrees awarded). Current student data systems yield little information about an institutions educational effectiveness --the outcomes it produces. Such questions as "What and how much do students learn from any given postsecondary education experience?" go largely unaddressed by current student data systems. To answer such questions, one must look beyond this operational perspective to examine current and projected institutional, state, and national student outcome data needs and deficiencies in a policy context. The purpose of this report is to provide a model for examining, from a policy perspective, postsecondary education data priorities in the student outcomes area, as well as (at a later point in time) in other areas such as instructional costs, postsecondary access, workforce development, and new delivery systems.

The report first identifies a range of policy issues currently facing American higher education. NPECs "Student Outcomes from a Policy Perspective" Working Group developed the list. A taxonomy of student educational outcomes is then presented, and a procedure for linking the outcomes and policy taxonomies is presented as a first step in identifying those outcomes with the greatest potential for informing policy-making in postsecondary education. A set of criteria for selecting specific outcomes within each outcomes domain is then introduced, as is a process for applying those criteria. The report closes with a series of recommendations for future steps as work on the development of useful outcomes information for policy-making proceeds.

PDF Download/view the full report in a PDF file.(222K)

HELP Help with PDF files

For more information about the content of this report, contact Nancy Borkow at Nancy.Borkow@ed.gov.