• Introduction
• Objectives Covered in This Evaluation
• Overview of Findings and Recommendations
• Summary
• References

Academic libraries are education resources that play a key role in the transfer of knowledge and information. Data on these important resources are provided by the "Academic Libraries Survey" (ALS), one component of the Integrated Postsecondary Education Data System (IPEDS). ALS collects data on the libraries in the entire universe of accredited higher education institutions and on the libraries in nonaccredited institutions with a program of 4 years or more. The survey provides policymakers and researchers with information on trends in total operating expenditures devoted to academic libraries, on services available to students, and on the adoption of new technologies, such as electronic access to bibliographic information. In addition, ALS provides information on the staffing of academic libraries.

This evaluation was conducted for the National Center for Education Statistics (NCES) by the Governments Division of the U.S. Census Bureau. It utilizes five distinct categories to evaluate ALS coverage: (1) policy, (2) survey design and data elements, (3) universe of participants, (4) coordinator interviews, and (5) public versus private institution reporting.

Policy is examined in terms of National Performance Review guidelines for "Best Practices" in government research and the importance of this survey in the complex environment associated with our National Information Infrastructure. Second, survey design is assessed to evaluate what kind of data are covered by ALS and how they measure up to professional standards set by the American Library Association (ALA) and other notable academic library research groups. Third, universe coverage is evaluated by comparing the list of IPEDS universe units to other lists applicable to academic library research. Sources for comparison were selected based on the professional respect that they command in the library field. Fourth, at the request of NCES, the evaluation examines the opinions of survey coordinators regarding instrument design and data covered by ALS. Finally, taking coverage a step further, private versus public institution reporting is examined.

Instrument revisions are consistent with national performance objectives established by the Office of Management and Budget and the National Performance Review. As a result, IPEDS is creating a more efficient and effective instrument for data collection. By incorporating a reader-focused environment, erroneous reporting should be decreased, and editing (data cleaning) time should theoretically be reduced as well. Therefore, coverage quality should be increased or maintained based on policy and survey design initiatives currently in place.

Due to the size of the ALS universe, data gathering is extremely tedious and complex. Only those institutions that receive federal funding through Title IV student financial aid programs¹ are required to respond to the survey. Regardless of the presence or absence of federal funding, institution response is initiated within a self-paced environment. Electronic software and Web technologies are helping to reduce the time it takes for the institution to respond to ALS. As a mutual support mechanism to alleviate the timeliness issue, an early release policy for the data is envisioned. It is possible that, by reducing the time necessary for data collection, data dissemination could occur at an earlier date as well. If achieved, efficient reporting could directly affect the timeliness issue associated with data dissemination as well. Institutions that have the option of participating in ALS might elect to do so in light of these changes.

Field coordinators collect ALS data from the institutions in their regions, then submit these data either by using IDEALS electronic reporting software or by returning the actual survey forms filled out by the institutions. Field coordinators are an excellent resource to assess the quality of institution coverage and instrument design. This evaluation proposes that a short questionnaire be included in the IDEALS software to assess this valuable resource for longitudinal and cross-sectional evaluation of ALS. By utilizing their firsthand experience, library representatives could help NCES maintain or increase the quality of data coverage and collection at the regional level.

The quality of institutional coverage remains excellent when compared to other institutional listings directly related to the academic libraries industry. Seven reputable listing types were compared:

Data type	Universe assembled by
Branches	American Library Directory (ALD) (1997-98, Volume 1)
Library Science Programs	American Library Directory (ALD) (1997-98, Volume 2) American Library Association (Accredited Library Science Programs, as listed in ALD, Volume 2)
Associations	Association of Research Libraries The Oberlin Group
World Wide Web	University of Florida's Web Listing of Access Colleges and Universities
Archives, Special Collections, and Archival Education Programs	U.S. Government Printing Office Web Listing of Federal Depositions University of Idaho/Abraham's Listing of Special Collections The Society of American Archivists
Professional School Libraries	American Bar Association List of Approved Law Schools
General	Peterson's Guides

Findings suggest that the ALS universe is superior (coverage gap of 1 to 3 percent in only two of the listings).² Regardless of this finding, future studies are needed to assess whether or not the data collected by ALS fully account for branch data associated with parent institution resources. The only resource that could come close to assessing this quality would be branch data compiled from the American Library Directory (1997).

A problem currently plaguing ALS data is uncertainty as to the presence or absence of professional school statistics in parent college or university data. Branch comparison could be valuable in light of this problem as well. In an effort to clarify parent institution reporting, the instrument could include questions indicating whether or not professional school resources are present or absent in aggregate institution statistics, a method already utilized by two professional academic library research associations, the Association of College and Research Libraries (ACRL) and the Association of Research Libraries (ARL).

Finally, based on the findings from the segment observation in this study (public versus private reporting), the most problematic institutional types associated with reporting would include the private, nonprofit, higher education, 4-year institutions (primarily of a religious affiliation). It is anticipated that the problem of nonresponse by Title IV institutions would be reduced if mechanisms to enforce participation were put into place nationally. The question remains: Where does ALS proceed from here?

Based on field coordinator response, ALS should continue to change along with the industry. Data coverage is a key factor in the assessment of institutional, regional, and national academic library resources. Without measuring current trends in procurement and management of resources, appropriations cannot be made to enhance resources and facilities that already exist.

Should resource statistics that do not pertain to "higher education institutions" in IPEDS data coverage be included in ALS reporting? Specifically, should library data for less-than-2-year institutions³ (primarily vocational and trade schools) be reported along with academic library data? It is already known that by definition these institutions fall outside the defined ALS universe (1998) of participants. Given trends in nontraditional education, for public officials to adequately assess library resources covered in a community or region, it might be necessary to include nontraditional library elements within the comprehensive sphere of resources available to areas and communities. Paralleling this argument, ALS field coordinators indicated in the national survey interview conducted as a part of this evaluation that vocational and nonacademic library resources do represent significant library resources in the United States.⁴ If the amount of data on these resources would be increased by coverage of nonacademic institutions, then this expanded coverage would further the U.S. Department of Education goal of identifying as many of the nation's education resources as possible.

Findings suggest that the data collected through ALS represent a high-quality product when compared to other surveys within the same field of study. Regardless of the problems with ALS outlined in this evaluation, it is the most comprehensive data source for academic libraries data of its kind in the United States. No other public or private association provides a more complete listing of resources offered by public and private colleges and universities. Because ALS data are functional in terms of policy assessment and resource allocation (funding), accurate statistics will provide for a more informed approach to planning and funding for academic libraries in the United States. Survey refinement and timely dissemination of ALS data will not only provide current statistics for the policymakers, but also provide a means for institutions to assess their own resources at the national, regional, and sector levels.