The descriptive comparisons were tested in this report using Student's t statistic. Differences between estimates are tested against the probability of a Type I error,⁶ or significance level. The significance levels were determined by calculating the Student's t values for the differences between each pair of means or proportions and comparing these with published tables of significance levels for two-tailed hypothesis testing.

Student's t values may be computed to test the difference between estimates with the following formula:

(1)

where E₁ and E₂ are the estimates to be compared and se₁ and se₂ are their corresponding standard errors. This formula is valid only for independent estimates. When estimates are not independent, a covariance term must be added to the formula:

(2)

where r is the correlation between the two variables.⁷ The denominator in this formula will be at its maximum when the two estimates are perfectly negatively correlated, that is, when r = –1. This means that a conservative dependent test may be conducted by using –1 for the correlation in this formula as follows:

(3)

The estimates and standard errors are obtained from the DAS. If the comparison is between the mean of a subgroup and the mean of the total group, the following formula is used:

(4)

where p is the proportion of the total group contained in the subgroup.⁸ The estimates, standard errors, and correlations can all be obtained from the DAS.

There are hazards in reporting statistical tests for each comparison. First, comparisons based on large t statistics may appear to merit special attention. This can be misleading since the magnitude of the t statistic is related not only to the observed differences in means or percentages but also to the number of respondents in the specific categories used for comparison. Hence, a small difference compared across a large number of respondents would produce a large t statistic.

A second hazard in reporting statistical tests for each comparison occurs when making multiple comparisons among categories of an independent variable. For example, when making paired comparisons among different levels of income, the probability of a Type I error for these comparisons taken as a group is larger than the probability for a single comparison. When more than one difference between groups of related characteristics or "families" are tested for statistical significance, one must apply a standard that assures a level of significance for all of those comparisons taken together.

Comparisons were made in this report only when p < .05/k for a particular pairwise comparison, where that comparison was one of k tests within a family. This guarantees both that the individual comparison would have p < .05 and that for k comparisons within a family of possible comparisons, the significance level for all the comparisons will sum to p < .05.⁹

For example, in a comparison of the percentages of males and females who attend research and doctoral public institutions, only one comparison is possible (males versus females). In this family, k=1, and the comparison can be evaluated without adjusting the significance level. When respondents are divided into three income groups and all possible comparisons are made, then k=3 and the significance level of each test must be p< .05/3, or p< .017. The formula for calculating family size (k) is as follows:

(5)

where j is the number of categories for the variable being tested. In the case of income, there are three groups (low quartile, middle two quartiles combined, and high quartile), so substituting 3 for j in equation 5,