Trends in the Use of School Choice: 1993 to 2007
NCES 2010-004
April 2010

Statistical Tests

All specific statements of comparisons have been tested for statistical significance at the .05 level using Student's t-statistics to ensure that the differences are larger than those that might be expected due to sampling variation. Adjustments for multiple comparisons were not included. Readers are cautioned not to draw causal inferences based on the univariate and bivariate results presented. Many of the variables examined in this report may be related to one another, and complex interactions and relationships among the variables have not been explored. The variables examined here are also just a few of those that can be examined in these data.

The tests of significance used in this report are based on Student's t statistics for the comparisons of means and of percentages. To test for a difference between two subgroups in the population percentage having a particular characteristic, say p1 versus p2, the test statistic is computed as:

text statistic equation

where pi is the estimated percentage of subgroup i(i = 1, 2) having the particular characteristic and s.e.(pi) is the standard error of that estimate. Thus, if p1 is the 74 percent of students attending assigned public schools in 2003, with a standard error of 0.6, and p2 is the 73 percent of students attending assigned public schools in 2007, with a standard error of 0.7, then the t-value is equal to 0.74. The decision rule is to reject the null hypothesis (i.e., there is no measurable difference between the two groups in the population in terms of the percentage having the characteristic) if |t| > tα/2;df, where tα/2;df is the value such that the probability a Student's t random variable with df degrees of freedom exceeds that value is α/2 . All tests in this report are based on a significance level of 0.05, i.e., α = 0.05. When the degrees of freedom are large, greater than 120, t0.025;df ≈ 1.96. Regarding the example given above, the t-value of 0.74, which is less than 1.96, indicates that the null hypothesis cannot be rejected. Simply put, there is no statistically measurable difference between the percentage of students attending assigned public schools in 2003 compared with 2007.

Tests of significant differences in estimates for students in assigned public schools across more than two years are based on regressions. Regression tests were used to verify that there was a positive or negative trend over time. We made one exception to discussing trends for only assigned public schools. We tested the overall trend in percentage enrollment for both types of private schools and chosen public schools with a regression for summary statements. However, in the detailed findings by subpopulations, we discuss only point-to-point comparisons.