Technical Notes: Sampling Errors

The responses were weighted to produce national estimates (table B-1). The weights were designed to reflect the variable probabilities of selection of the sampled institutions and were adjusted for differential unit (questionnaire) nonresponse. The nonresponse weighting adjustments were made within classes defined by variables used in sampling and expected to be correlated with response propensity: instructional level, control, highest level of offering, and total enrollment. Within the final weighting classes, the base weights (i.e., the reciprocal of institutions' probabilities of selection) of the responding institutions were inflated by the inverse of the weighted response rate for the class. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability. Jackknife replication was used to estimate the sampling variability of the estimates and to test for statistically significant differences between estimates.

Because the data from the PEQIS survey on dual enrollment programs and courses were collected using a complex sampling design, the variances of the estimates from this survey (e.g., estimates of proportions) are typically different from what would be expected from data collected with a simple random sample. Not taking the complex sample design into account can lead to an under- or overestimation of the standard errors associated with such estimates (Kish 1965). To generate accurate standard errors for the estimates in this report, standard errors were computed using a technique known as jackknife replication (Levy and Lemeshow
1991). A form of jackknife replication referred to as the JKN method was used to construct the replicates. Under the JKN method, the replicates were formed within groups of institutions (called "variance strata) within which institutions were sampled at approximately the same rate. By creating the jackknife replicates within the variance strata, finite population correction factors (FPCs) can be introduced in the variance estimator to account for the fact that institutions in some variance strata were sampled at relatively high rates (Rust 1986, Wolter 1985). The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. A total of 100 jackknife replicates was created for variance estimation. A computer program (WesVar) was used to calculate the estimates of standard errors.12

The standard error is a measure of the variability of an estimate due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example the estimated percentage of degree-granting postsecondary institutions with a dual enrollment program is 46 percent and the standard error is 0.8 percent (tables 1 and 1a). The 95 percent confidence interval for the statistic extends from [46 – (0.8 x 1.96)] to [46 + (0.8 x 1.96)], or from 44.4 to 47.6 percent. The 1.96 is the critical value for a two-sided statistical test at the p < .05 significance level (where .05 indicates the 5 percent of all possible samples that would be outside the range of the confidence interval).

Comparisons can be been tested for statistical significance at the p < .05 level using Student's t-statistic to ensure that the differences are larger than those that might be expected due to sampling variation. Student's t values are computed to test the difference between estimates with the following formula:

where E₁ and E₂ are the estimates to be compared and se₁ and se₂ are their corresponding standard errors.

---

12 The WesVar program and documentation is available for download at http://www.westat.com/Westat/expertise/information_systems/WesVar/index.cfm.