Table of Contents  |  Search Technical Documentation  |  References

t test for Two Independent Groups

In NAEP, a t test for independent samples is used to compare population means where there is no overlap in terms of sampled students representing these populations. t tests are conducted to determine whether the mean of a student population is significantly different from that of another population (e.g. male vs. female students) given the sample means and the standard errors attached to those means.

Let Ai be the statistics in question (e.g., the mean for group i) and let S(Ai) be the standard error of the statistic. The estimates for groups i and j are said to be significantly different so that the two groups can be considered to come from two different populations if

the absolute value of A sub i minus A sub j over the sqaure root of S parens A sub i close parens squared plus S parens A sub j close parens squared all greater than or equal to T sub alpha over 2

where T sub alpha over 2  is the 1 minus the fraction alpha over 2 percentile of the t distribution with df degrees of freedom and α equal to 0.05. As mentioned above, the assumption is made that the results being compared are based on data from two independent samples. However, because a multi-stage sampling design is used where students are located within schools and schools within geographic regions, students from the groups compared are not strictly independent. For instance, male and female students in a classroom tend to have similar responses as they tend to have similar learning experiences and backgrounds. Hence, there is usually a positive sample dependency between these two groups. Subsequently, independent t tests could be considered conservative in this case in the sense that the number of significant results is underreported.

Last updated 19 March 2009 (GF)

Printer-friendly Version