Skip Navigation

Academic Libraries Survey (ALS)



5. Data Quality and Comparability

NCES makes every effort to achieve high data quality. Through a web collection that includes built-in edit checks, it hopes to improve the quality of ALS data. Users are cautioned about limitations in the analysis of ALS data by state or by level and control of institution. Since nonresponse varies by state, the reliability of state estimates and comparisons is affected. Special caution should be exercised when using data where the nonresponse rate is 15 percent or greater. The procedure of using medians instead of means for imputation also represents a change from past survey cycles, and while research indicates that the effect of the change in imputation procedure is not large, caution should be exercised in making comparisons with 2000 or earlier reports. See below for more information on the types of errors that affect data quality and comparability.

Between 2000 and 2012, the ALS collected data independent from the IPEDS data collection; however, data from the ALS could still be linked to IPEDS data using the institution’s unique unit identification number (UNITID). IPEDS serves as the frame, or universe, of degree-granting postsecondary institutions from which eligible institutions are selected for the current ALS administration.

Sampling Error

Because ALS is a universe survey, there is no sampling error.

Nonsampling Error

Coverage Error. A comprehensive evaluation of the coverage of ALS found that the quality of institutional coverage was excellent (a coverage gap of only 1 to 3 percent) when compared to other institutional listings directly related to the academic libraries industry; however, questions remain as to whether the data collected by ALS fully account for branch data associated with parent institution resources. A second problem is that the ALS data for some parent colleges or universities may not contain statistics for their professional schools.

Nonresponse Error.
Unit nonresponse. The overall unit response rate for the 1996 ALS was 94.5 percent, or approximately 3,700 out of 3,900 possible institutions. Beginning in 1998, the ALS was limited to 2- and 4-year degree granting institutions. Of approximately 4,100 degree-granting postsecondary institutions in the United States in 1998, there were 160 institutions that did not have their own library, and those institutions were excluded. There were roughly another 300 institutions that were found to be out of scope due to not having an academic library based on the survey definition. Among the remaining 3,700 degree-granting postsecondary institutions in the 50 states and the District of Columbia in 1998, the overall unit response rate was 97 percent. For detailed unit response rates, see table ALS-1.

Of roughly 3,900 postsecondary institutions in the 50 states and District of Columbia that were identified to be surveyed for the 2000 ALS, approximately 160 were identified as “child” institutions and another 240 were excluded due to being out of scope; data for the child institutions were provided by the parent institution or office, and out-of-scope institutions were defined based on their responses to screening questions. Institutions were defined as out-of-scope if: 1) they indicated they did not participate in Title IV funding; 2) they indicated they had no library at the institution; or, 3) they did not respond to the screening questions. Of the remaining 3,500 in-scope and non-child degree-granting Title IV recipient, postsecondary education institutions, approximately 3,100 responses were received yielding an overall unit response rate of 87.4 percent.

For the 2002 ALS, 3,900 postsecondary institutions were identified. Of these, approximately 170 were child institutions and another 150 were excluded due to being out of scope; the remaining 3,600 degree-granting postsecondary institutions had a response rate of 88.6 percent.

In 2004, there were approximately 4,100 possible postsecondary institutions, of which 240 were child institutions and 220 were out of scope. For the remaining 3,700 institutions, the overall unit response rate was 87.0 percent.

For the 2006 ALS, of the roughly 4,100 postsecondary institutions, there was an overall unit response rate of 88.8 percent out of 3,600 in-scope, non-child, degree-granting Title IV-receiving postsecondary institutions. Roughly 230 child institutions were excluded, with the remaining difference being primarily attributable to out-of-scope institutions.

The overall unit response rate for the 2008 ALS was 86.7 percent of the 3,800 in-scope, non-child, degree granting Title IV recipient postsecondary institutions; 250 child institutions were excluded, with additional out-of-scope institutions being excluded from the 4,200 possible.

Top

Table ALS-1. Unit response rates by year and highest level of degree offered: Selected fiscal years 1996–2012

  Highest level of degree offered 1996¹ 1998 2000 2002 2004 2006 2008 2010 2012
  Overall response rate 94.5 97.0 87.4 88.6 87.0 88.8 86.7 86.1 85.3
Less than 4-year 95.8 85.8 86.6 84.3 86.7 86.1 84.5 84.0
4-year 97.7 88.5 89.8 88.8 90.0 87.1 87.1 86.0
  Doctor’s 98.8 91.0 91.0 91.3 91.7 89.3 92.2 90.8
  Master’s 97.0 89.5 89.6 89.0 89.6 91.0 91.0 86.9
  Bachelor’s 97.9 85.5 89.5 86.5 89.2 80.0 76.4 79.3
— Not available.
¹Only the overall unit response rate was available for the 1996 survey.
SOURSE: Data File and Documentation, Public Use: Academic Libraries Survey (ALS): Fiscal Year 1996, Holton & George 2007, p. 7; Academic Libraries: 1998, Cahalan & Justh, p. 62; Documentation for the Academic Library Survey (ALS) Data File: Fiscal Year 2000 Public Use, Schmitt 2006, p. F-2; Documentation for the Academic Library Survey (ALS) Data File: Fiscal Year 2002, Schmitt 2005, p. F-2; Documentation for the Academic Library Survey (ALS) Data File: Fiscal Year 2004 Public Use, Schmitt et al. 2007, p. F-2; Documentation for the Academic Library Survey (ALS) Public Use Data File Fiscal Year 2006, Holton et al. 2008, p. 57; Documentation for the Academic Library Survey (ALS) Public Use Data File: Fiscal Year 2008, Phan et al. 2009, p. F-3; Documentation for the Academic Libraries Survey (ALS) Public Use Data File: Fiscal Year 2010, Phan et al. 2011b, p. F-3; Academic Libraries: 2012, Phan et al. 2014, p. 24

There were a total of 3,700 ALS-eligible institutions for the 2010 survey, with a response rate of 86.1 percent. Excluded from the 4,100 possible postsecondary institutions were 300 child institutions plus additional out-of-scope institutions, thus leaving 3,700 eligible in-scope, non-child, degree granting Title IV recipient postsecondary institutions.

For the 2012 survey, there were approximately 3,800 degree-granting postsecondary institutions in the 50 states and the District of Columbia that had academic libraries in 2012. The response rate among these institutions was approximately 85.3 percent.

Item Nonresponse. For the 1998 ALS, item response rates ranged from 66.0 to 94.0. For the 2000 ALS, most item response rates ranged from 68.6 to 86.9 percent. For 2004, overall item response rates ranged from 73.4 to 86.7 percent. Overall item response rates in 2006 ranged from 78.9 to 88.8 percent, while overall item response rates in 2008 ranged from 71.8 to 86.3 percent. For 2010, item response rates ranged from 69.3 to 85.3 percent, and for 2012 the range was 73.2 to 83.9 percent.

Measurement Error. Based on preliminary analysis conducted using the 2010 ALS data collection, there is evidence of measurement error due to the survey questionnaire’s design. For example, it is not clear how universities on a quarter system report for the “Fall” reference period referred to in the survey instructions. Since the reference period is in the instructions which are contained in a separate section of the questionnaire, and since instructions are frequently not read by respondents, the respondents may not be reporting for the correct time period.

Top