Quick Response Information System: Postsecondary Education Quick Information System (PEQIS)

5. DATA QUALITY AND COMPARABILITY

Sampling Error

Estimates are based on the selected samples and, consequently, are subject to sampling variability. The standard error is a measure of the variability of estimates due to sampling. Because the data from PEQIS surveys are collected using a complex sampling design, the variances of the estimates from the surveys (e.g., estimates of proportions) are typically different from what would be expected from data collected with a simple random sample. To generate accurate standard errors for the estimates, standard errors are computed using a technique known as jackknife replication

Nonsampling Error

Nonsampling error describes variations in the estimates that may be caused by population coverage limitations and data collection, processing, and reporting procedures. The sources of nonsampling errors are typically problems like unit and item nonresponse, differences in respondents’ interpretations of the meaning of questions, response differences related to the particular time the survey was conducted, and mistakes made during data preparation. It is difficult to identify and estimate either the amount of nonsampling error or the bias caused by this error.

To minimize the potential for nonsampling error, PEQIS surveys use a variety of procedures, including a pretest of the questionnaire with the individual at each postsecondary institution deemed to be the most knowledgeable about the survey topic. The pretest provides the opportunity to check for consistency in the interpretation of questions and definitions and to eliminate ambiguous items. The questionnaire and instructions are also extensively reviewed by NCES and the data requestor. In addition, both range and logic editing of the questionnaire responses is conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items are contacted by telephone to resolve problems. Data are keyed with 100 percent verification for surveys received by mail, fax, e-mail, or telephone.

Coverage error. Because the sampling frames for PEQIS surveys are constructed from IPEDS data files, coverage error is believed to be minimal.

Nonresponse error. Both unit nonresponse and item nonresponse are quite low in PEQIS surveys. Among recent PEQIS surveys, weighted response has ranged from 87 to 95 percent (see table PEQIS-1). Item nonresponse for most items in PEQIS surveys has been less than 1 percent. The weights are adjusted for unit nonresponse. Imputation is performed for item nonresponse.

Measurement error. This type of nonsampling error may result from different interpretations of survey definitions by respondents or from the institution's inability to report certain data due to its recordkeeping system. Nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used. These types of experiments are not generally conducted by PEQIS.

Top

Comparability

While most PEQIS surveys are not designed specifically for comparison with other surveys, the data from some PEQIS surveys can be compared with data from other postsecondary surveys. For example, the 1998 Survey on Students With Disabilities at Postsecondary Education Institutions complements another NCES study on the self-reported preparation, participation, and outcomes of students with disabilities. In another example, the 1995 Survey on Remedial Education in Higher Education Institutions can be compared to data from remedial education surveys conducted by the American Council on Education and the Southern Regional Education Board. A third example is the survey on Services and Support Programs for Military Service Members and Veterans: 2012-13, which can be compared to the 2012 American Council on Education survey on campus programs for veterans and service members.

In addition, some PEQIS surveys have been repeated so that results can be compared over time. Examples of these surveys are listed below.

• The PEQIS survey on dual enrollment programs and courses for high school students was conducted for the 2002–03 and 2010–11 academic years.
• The PEQIS survey on students with disabilities at postsecondary institutions was conducted for 1998 and the 2008–09 academic years.
• PEQIS collected data on distance education at postsecondary institutions in 1995, 1998–99, and 2000–01.¹
• PEQIS conducted surveys on remedial education in 1995 and 2000.²

footnotes
¹A fourth PEQIS survey on distance education, conducted in 2006–07, included many of the same topics covered in the previous surveys, but the data are not comparable because of the revised definition of distance education.
²Two earlier surveys on remedial education in postsecondary institutions, for academic years 1983–84 and 1989–90, were conducted using the FRSS because PEQIS was not yet in existence.

Table PEQIS-1. Sample sizes and weighted response rates for recent PEQIS surveys: Selected years, 2002–2013
Survey	Sample size	Weighted response rate
PEQIS 19: Services and Support Programs for Military Service Members and Veterans, 2012–13	1650	90
PEQIS 18: Dual Enrollment Programs and Courses of High School Students, 2010–11	1650	94
PEQIS 17: Students with Disabilities at Postsecondary Education Institutions, 2008–09	1558	89
PEQIS 16: Online and Distance Education at Postsecondary Institutions, 2006–07	1628	87
PEQIS 15: Educational Technology and Teacher Education Programs for Initial Licensure, 2006	2512	95
PEQIS 14: Dual Enrollment Programs and Courses for High School Students, 2002–03	1610	92
SOURCE: U.S. Department of Education, ED Data Inventory. Available at https://datainventory.ed.gov.