A.6 Bias Analysis

This analysis is based on 14,710 eligible students who had participated in the base-year interview and the first follow-up interview. Of these, 730 did not participate in the BY mathematics assessment and 650 did not participate in the F1 mathematics assessment. The final analytic sample includes 9,460 respondents, of whom 5,300 have course sequences followed by more than 200 respondents—referred to here as those with designated mathematics course sequences. A bias analysis was conducted to assess the generalizability of the final analysis sample (n = 9,460) and those with designated course sequences (n = 5,300) compared with the target population of sophomores who participated in both the base-year interview and first follow-up interview (n = 14,710). Table A-13 shows the distributions of the student and school characteristics used in this study for each of the three samples, weighted using the panel weight (F1PNLWT).

Using the 5 percentage point threshold as a criterion for meaningful differences, the analytic sample and the analytic sample with designated course sequences differs from the target population in two ways: compared with the target population, the analytic sample and the analytic sample with designated course sequences include a higher proportion of students who are White and a higher proportion of students who expect a bachelor's degree or higher. This is not surprising as the analysis sample excludes students who had transferred, students who were absent on the day of the test administration, and students with incomplete transcripts. This accords with other research which shows that racial/ethnic minorities are more likely to drop out relative to White students (U.S. Department of Education 1999) and that transfer behavior is associated with dropping out (Rumberger and Larson 1998)—both potentially contributing to higher rates of incomplete transcript information among these groups (Ingels et al. 1995). Additionally, compared with the full sophomore panel, the sample of those with designated mathematics course sequences contains more students living with both their father and their mother. Therefore, the analytic sample does not entirely approximate the composition of the full sophomore panel. Despite these differences, it is imperative to have complete transcript information and to have both unimputed mathematics achievement test scores (in the base year and the first follow-up) to accurately answer the research questions posed in this report. A consequence of using the analytic sample is that the findings may not generalize to all students, particularly those who are non-White, those who have educational expectations that do not include college completion, and those who are not living with their mother and father. Readers should keep this caveat in mind when interpreting the results.