Skip Navigation
Remedial Education at Higher Education Institutions in Fall 1995
NCES 97584
October 1996

Survey Methodology and Data Reliability

Postsecondary Education Quick Information System

The Postsecondary Education Quick Information System (PEQIS) was established in 1991 by the National Center for Education Statistics, U.S. Department of Education. PEQIS is designed to conduct brief surveys of postsecondary institutions or state higher education agencies on postsecondary education topics of national importance. Surveys are generally limited to two or three pages of questions, with a response burden of about 30 minutes per respondent. Most PEQIS institutional surveys use a previously recruited, nationally representative panel of institutions. The sampling frame for the PEQIS panel recruited in 1992 was constructed from the 1990-91 Integrated Postsecondary Education Data System (IPEDS) Institutional Characteristics file. Institutions eligible for the PEQIS frame for the panel recruited in 1992 included 2-year and 4-year (including graduate-level) institutions (both institutions of higher education and other postsecondary institutions), and less-than-2-year institutions of higher education located in the 50 states, the District of Columbia, and Puerto Rico: a total of 5,317 institutions.

The PEQIS sampling frame for the panel recruited in 1992 was stratified by instructional level (4-year, 2-year, less-than-2-year), control (public, private nonprofit, private for-profit), highest level of offering (doctor's/first professional, master's, bachelor's, less than bachelor's), total enrollment, and status as either an institution of higher education or other postsecondary institution. Within each of the strata, institutions were sorted by region (Northeast, Southeast, Central, West), whether the institution had a relatively high minority enrollment, and whether the institution had research expenditures exceeding $1 million. The sample of 1,665 institutions was allocated to the strata in proportion to the aggregate square root of full-timeequivalent enrollment. Institutions within a stratum were sampled with equal probabilities of selection. During panel recruitment, 50 institutions were found to be ineligible for PEQIS, primarily because they had closed or offered just correspondence courses. The final unweighted response rate at the end of PEQIS panel recruitment in spring 1992 was 98 percent (1,576 of the 1,615 eligible institutions). The weighted response rate for panel recruitment was 96 percent.

Each institution in the PEQIS panel was asked to identify a campus representative to serve as survey coordinator. The campus representative facilitates data collection by identifying the appropriate respondent for each survey and forwarding the questionnaire to that person.

Sample and Response Rates

The sample for this survey consisted of two-thirds of the 2-year and 4-year (including graduate-level) higher education institutions in the PEQIS panel, for a sample of 849 institutions. In late September 1995, questionnaires (see appendix B) were mailed to the PEQIS coordinators at the institutions. Coordinators were told that the survey was designed to be completed by the person at the institution most knowledgeable about the institution's remedial education courses.

Two institutions were found to be out of the scope of the survey because they were closed, leaving 847 eligible institutions. These 847 institutions represent the universe of approximately 3,450 2-year and 4-year (including graduate-level) higher education institutions in the 50 states, the District of Columbia, and Puerto Rico, including institutions that do not enroll freshmen. Telephone followup of nonrespondents was initiated in late October 1995; data collection and clarification was completed in late January 1996. For the eligible institutions that received surveys, an unweighted response rate of 94 percent (797 responding institutions divided by the 847 eligible institutions in the sample) was obtained. The weighted response rate for this survey was 96 percent. The unweighted overall response rate was 92 percent (97.6 percent panel recruitment participation rate multiplied by the 94.1 percent survey response rate). The weighted overall response rate was 92 percent (96.1 percent weighted panel recruitment participation rate multiplied by the 95.7 percent weighted survey response rate).

Weighted item nonresponse rates ranged from 0 percent to 4.9 percent, except for retention of freshmen enrolled in remedial courses, which had a nonresponse rate of 8.5 percent. Item nonresponse rates for most items were less than 1 percent. Because the item nonresponse rates were so low, imputation for item nonresponse was not implemented.

Sampling and Nonsampling Errors

The response data were weighted to produce national estimates (see Table 16). Since all analyses of the data included only those institutions that enroll freshmen, the number of respondents and the national estimates presented in Table 16 represent higher education institutions that enroll freshmen, rather than all higher education institutions. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability.

The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in data collection. These errors can sometimes bias the data. Nonsampling errors may include such problems as misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

To minimize the potential for nonsampling errors, the questionnaire was pretested with respondents at institutions like those that completed the survey. During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were extensively reviewed by the National Center for Education Statistics and the Office of the Under Secretary, U.S. Department of Education. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of institutions reporting that the institution offered at least one remedial reading, writing, or mathematics course is 77.6 percent, and the estimated standard error is 1.7 percent. The 95 percent confidence interval for the statistic extends from [77.6 -(1.7 times 1.96)] to [77.6 + (1.7 times 1.96)], or from 74.3 to 80.9 percent. Tables of standard errors for each table and figure in the report are provided in appendix A.

Estimates of standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variances of the statistics.8 To construct the replications, 51 stratified subsamples of the full sample were created and then dropped one at a time to define 51 jackknife replicates.8 A computer program (WesVarPC), distributed free of charge by Westat, Inc., through the Internet, was used to calculate the estimates of standard errors. WesVarPC is a stand-alone Windows application that computes sampling errors for a wide variety of statistics (totals, percents, ratios, log-odds ratios, general functions of estimates in tables, linear regression parameters, and logistic regression parameters).

The test statistics used in the analysis were calculated using the jackknife variances and thus appropriately reflected the complex nature of the sample design. In particular, an adjusted chi-square test using Satterthwaite's approximation to the design effect was used in the analysis of the two-way tables.9 Finally, Bonferroni adjustments were made to control for multiple comparisons where appropriate. For example, for an "experiment-wise" comparison involving g pairwise comparisons, each difference was tested at the 0.05/g significance level to control for the fact that g differences were simultaneously tested.

Comparison With Other Estimates of Remedial Education

The percentage of institutions offering remedial education as estimated by this PEQIS survey is about the same as the percentage of institutions offering remedial instructional services as estimated from the Integrated Postsecondary Education Data System (IPEDS) Institutional Characteristics Survey. This PEQIS survey estimated that 78 percent of higher education institutions enrolling freshmen offered at least one remedial reading, writing, or mathematics course in fall 1995. IPEDS estimated that 79 percent of higher education institutions offered remedial instructional services in academic year 1993-94.10

Estimates of the percentage of students in remedial education vary depending upon whether data are reported by institutions or are student self-reports. This PEQIS survey, which uses data reported by institutions, estimated that 29 percent of first-time freshmen were enrolled in remedial reading, writing, or mathematics, with estimates of 13 percent enrolled in reading, 17 percent in writing, and 24 percent in mathematics. Data from other institutional surveys are fairly similar to the PEQIS estimates. A recent study by the American Council on Education (ACE) found that an average of 33 percent of first-year undergraduates needed remedial work in mathematics and 27 percent needed remedial work in English in fall 1994.11 An earlier study conducted by the Southern Regional Education Board (SREB) in member states estimated that an average of 36 percent of first-time freshmen needed remedial instruction in 1986.12 It should be noted that the ACE and SREB studies asked about the percentage of freshmen needing remediation and not about the percentage of freshmen actually enrolled in remedial courses.

Estimates of remedial enrollments from student self-reports are quite different. The 1993 National Postsecondary Student Aid Study estimated that 17 percent of first-year undergraduates at higher education institutions reported taking remedial courses during academic year 1992-93, with estimates of 7 percent for reading, 6 percent for writing, and 10 percent for mathematics.13

Postsecondary transcripts provide another way to examine remedial enrollments. A recent publication from the U.S. Department of Education (The New College Course Map14 ) examined course taking by postsecondary students based on the transcripts collected during two national longitudinal studies. The National Longitudinal Study of the High School Class of 1972 provided information about student course taking at postsecondary institutions from 1972 to 1984. The High School and Beyond/Sophomores study provided information about postsecondary experiences from 1981 to 1993. According to The New College Course Map, the percentage of students earning postsecondary credits in all remedial courses (English, precollegiate mathematics, and basic study skills) has stayed constant across both studies at 46 percent of students who earned more than 10 postsecondary credits. This report also notes that community colleges provide the bulk of remedial courses.

There are many differences in methodologies and populations of interest in these various studies. In particular, the NPSAS estimates were student self-reports, while the other data were obtained from institutional respondents and records. Since the PEQIS study was designed to obtain estimates from institutional respondents about remedial enrollments and was not designed as a comparative study, the reasons for differences in the estimates from these various sources cannot be answered with the available data.

Background Information

The survey was performed under contract with Westat, Inc., using the Postsecondary Education Quick Information System (PEQIS). This is the sixth PEQIS survey to be conducted. Westat's Project Director was Elizabeth Farris, and the Survey Manager was Laurie Lewis. Bernie Greene was the NCES Project Officer. The data were requested by the Planning and Evaluation Service, Office of the Under Secretary, U.S. Department of Education.

This report was reviewed by the following individuals:

Outside NCES

  • Elizabeth Eisner, Planning and Evaluation Service, Office of the Under Secretary, U.S. Department of Education
  • Kent Phillippe, American Association of Community Colleges
  • Meredith Ludwig, American Association of State Colleges and Universities

Inside NCES

  • Michael Cohen, Statistical Standards and Services Group
  • Mary Frase, Data Development and Longitudinal Studies Group
  • James Houser, Data Development and Longitudinal Studies Group
  • Paula Knepper, Data Development and Longitudinal Studies Group
  • Edith McArthur, Data Development and Longitudinal Studies Group
  • Peter Stowe, Surveys and Cooperative Systems Group

For more information about the Postsecondary Education Quick Information System or the Survey on Remedial Education in Higher Education Institutions, contact Bernie Greene, Data Development and Longitudinal Studies Group, National Center for Education Statistics, Office of Educational Research and Improvement, 555 New Jersey Avenue, NW, Washington, DC 20208-5651, telephone (202) 219- 1366.


8 Ibid, 183.

9 For example, see D. Rao and A. Scott. "On Chi-square Tests for Multi-way Contingency Tables with Cell Proportions Estimated from Survey Data," Annals of Statistics 12 (1984): 46-60.

10 U.S. Department of Education, National Center for Education Statistics. Characteristics of the Nation's Postsecondary Institutions: Academic Year 1993-94. NCES 94-388. 1994. Washington, DC.

11 Elaine El-Khawas. Campus Trends 1995. American Council on Education. 1995.
Washington, DC.

12 Ansley Abraham. Remedial Education in College: How Widespread Is It? No. 24. Southern
Regional Education Board. 1988. Atlanta, Georgia.

13 14 U.S. Department of Education, National Center for Education Statistics. 1993 National
Postsecondary Student Aid Study, unpublished tabulations, June 1996.

14 Clifford Adelman. The New College Course Map and Transcript Files. U.S. Department of
Education. 1995. Washington, D.C.

Top