• Introduction
• Background for Part I: Duplication of Data Collection
• Background for Part II: Definitional Analysis of Income and Expenditures Items
• Recommendations

This report was prepared for the National Center for Education Statistics (NCES) by the Governments Division of the U.S. Bureau of the Census. The report documents an evaluation that focused on the fiscal section of the NCES State Library Agencies (STLA) Survey,¹ Parts K, L, and M. The evaluation had two objectives: (1) to check for duplication of data collection between the survey and the compliance materials of the Office of Library Programs (OLP), a former agency of the U.S. Department of Education; and (2) to analyze selected definitions.

Part I of this evaluation compares fiscal year (FY) 1995 data collected by the NCES survey with those collected by OLP, to determine whether OLP's Annual Financial and Performance Reports collected any of the data in items 154-163 or 190-200 of the NCES survey. The STLA Survey Steering Committee and NCES authorized this duplication analysis to address issues raised in an Office of Management and Budget memorandum written in 1994 (OMB No. 1850-0705). The memorandum had expressed concern that NCES and OLP may have been collecting the same data on Library Services and Construction Act (LSCA) funds,² thereby creating an excessive reporting burden for state library agencies (STLAs).

At the time this study began, Steering Committee and NCES officials were aware that to some extent there was an overlap in the collection of financial data between NCES and OLP. At the same time, they were aware that the two agencies collected the data in different groupings and that the data were collected according to different reporting periods. The duplication analysis was authorized to determine to what extent these and other differences prevented duplication and whether the differences were avoidable.

Part I of this study analyzes STLA data from FY 1995 only, for two reasons. First, because the forms used by each agency were essentially the same from FY 1994, when the survey began, through FY 1996, it was not necessary to scrutinize the data from all three years. Second, it was preferable to examine the most recent data possible, and at the time Part I began, the 1996 data were not yet complete.

The evaluation consulted the following reference sources for information about the survey and OLP materials: statutory and regulatory material, blank data collection instrument forms, instructions for form completion, actual respondent data, and phone interviews with selected respondents.

To understand this report, it is necessary to become familiar with the data collection objectives and methods of each agency. Because the main objective of each agency was different, the collection methods and data differed. A brief explanation of objectives and methods follows.

NCES publishes State Library Agencies (a publication tabulating and summarizing the NCES STLA survey) each year for STLA administrators, state legislators, and other policymakers. The purpose of the survey is to provide state and federal agencypolicymakers with information about STLAs. The data collected are useful to (1) the Chief Officers of State Library Agencies; (2) policymakers in the executive and legislative branches of federal and state governments; (3) government and library administrators at federal, state, and local levels; (4) the American Library Association and its members or customers; and (5) library and public policy researchers. The survey asks each STLA about the kinds of services it provides, its staffing practices, its collections, income and expenditure data, and more. Decisionmakers use the NCES survey to obtain information about services and fiscal practices. The NCES survey is conducted by the U.S. Census Bureau, under contract with NCES. The survey instrument is contained in a software program into which each STLA enters its responses electronically. The STLA may receive the program either on floppy disk or by e-mail. After completing the survey, the STLA sends it back to the data collection unit.

Until federal FY 1998, OLP was an agency within the Department of Education that funded the programs authorized by Titles I through III of the Library Services and Construction Act (LSCA).³ OLP solicited grant applications from the STLAs, determined the amount to be awarded to each, dispersed the funds, and monitored the programs administered by the STLAs and their subgrantees. When the Library Services and Technology Act of 1996 (LSTA) passed, it ordered that all federal library programs be transferred to the new Institute of Museum and Library Services (IMLS) after FY 1997. Unlike NCES, OLP did not compile data for publication and use by STLAs and other policymakers. Rather, OLP collected the LSCA information in order to evaluate the extent to which each STLA was adhering to the LSCA grant program regulations. The divergent objectives of the agencies have caused them to collect different data. NCES uses a broad focus, and collects information on many different aspects of STLA operation, only one of which is financial statistics. OLP had a much narrower scope, concentrating on each grant project and whether the related funds were expended properly.

Since the first NCES STLA Survey was conducted (FY 1994), some of the financial data (collected in Parts K, L, and M of the survey) returned by the respondents have been inconsistent and problematic. The Steering Committee and NCES requested this part of the evaluation in order to determine the causes of the anomalous data, and to revise the definitions for the affected items. The ensuing study relied on the survey forms, instructions, and the respondent survey manuals; actual respondent data from survey years 1994-96; and phone interviews with 28 respondents. While the phone interviews collected some important information, they also revealed an underlying problem: the respondents are too busy to spend much time completing the survey. As such, the respondents sometimes do not double-check the data they provide for obvious errors; they do not always investigate further if they do not understand a question or item definition; and they are often unable to explain why their data appear internally inconsistent. Those conditions made Part II a difficult prospect. Often, during phone interviews, the respondents were uncertain about why equations were out of balance. When this occurred, it became necessary to use circumstantial evidence to draw conclusions about the data. In other words, the evaluation analyzed the respondent's data reporting pattern over 3 years to piece together the respondent's interpretations of the survey items. The necessity of relying on circumstantial evidence affected the strength of some of the findings. However, those findings, together with a close examination of Parts K, L, and M, did allow for a revision of the relevant questions, definitions, and notes. The wording of suggested revisions took into account the fact that the Library Services and Construction Act (LSCA) will not be relevant to the FY 1998 survey. Instead, the Library Services and Technology Act (LSTA) will control the federal grants to STLAs. While it remains unknown exactly how STLA data will be collected under the LSTA regime, it is possible that the Institute of Museum and Library Services (IMLS) will collect fundamentally the same fiscal information as did OLP. Hence, the fiscal parts of the NCES survey may remain largely unaffected by the transfer of regimes. The suggested definitional modifications were constructed with this in mind. Minor editing was done where required, to reflect the evolution from LSCA to LSTA. References to "LSCA" were changed to "LSTA," and references to Titles I, II, and III were changed to "Section 231(a) [20 U.S.C. § 9141(a)] of the Library Services and Technology Act," the subsection that reflects the Act's two main focuses.

Because the data being collected by the NCES survey may be fundamentally similar to those that will be collected by the new Institute of Museum and Library Services (IMLS), Part I of this evaluation suggested that the Steering Committee might wish to consult with IMLS regarding the possibility of coordinating data collection in future years.

Because some STLAs were confused about how to measure income and how to classify expenditures, Part II suggested that the Steering Committee address these problems by making several changes and additions to survey questions, definitions, and notes. Details of these suggested revisions can be found in the complete report.

² LSCA is the federal law that formerly governed the creation and funding of programs for library development.

³ LSCA contained eight titles, each focusing on a particular area of concern to libraries and the public they serve. The most prominent of these were Titles I through III, which accounted for over 96 percent of LSCA funds.