Identifying early warning indicators in three Ohio school districts

Title:	Identifying early warning indicators in three Ohio school districts
Description:	The purpose of this study was to identify a set of data elements for students in grades 8 and 9 in three Ohio school districts that could serve as accurate early warning indicators of their failure to graduate high school on time and to comparatively examine the accuracy of those indicators. In order to identify the set of indicators with the optimal accuracy for each district, the research team collected student-level data on two cohorts of grade 8 and 9 students in each school district. Datasets used in the analyses included students’ four-year graduation status (the outcome) and 8th and 9th grade data on attendance, coursework, suspensions, and test score records (the candidate early warning indicators). Logistic regression and Receiver Operating Characteristic (ROC) curve analyses were used to identify the candidate indicators that were the consistent predictors of students’ failure to graduate on time in each district and to identify the cut points on the original scales that most accurately distinguish students who were at risk of not graduating on time from those who did graduate on time. The analyses were restricted to students who were first-time freshmen within the districts in 2006/07 or 2007/08, and excluded students who entered the district after grade 9. Students in the 2006/07 cohort graduated in 2010, and students in the 2007/08 cohort graduated in 2011. The three districts included in the study varied in size, demographic composition, and locale. Results show that the optimal cut point for classifying students as at risk varied significantly across districts for five of the eight candidate indicators included in the study. Across the three districts and two grades, different indicators were identified as the most accurate predictors of students’ failure to graduate on time. End-of-year attendance rate was the only indicator that was a consistent predictor for both grades in all three districts. The most accurate indicators in both grade 8 and grade 9 were based on coursework (GPAs and course credits). Consistent with prior literature, failing more than one class and earning one or more suspensions also were strong predictors of failure to graduate on time. On average, indicators were more accurate in grade 9 than in grade 8. Findings illustrate why it is important for districts to conduct local validation using their own data to verify that indicators selected for their early warning systems accurately predict students’ failure to graduate on time. The methods laid out in this study can be used to help districts identify the best off-track indicators, and indicator cut points, for their particular early warning systems.
Online Availability:	Download, view and print the report as a pdf file. (394KB)
Cover Date:	July 2016
Web Release:	July 6, 2016
Print Release:	July 6, 2016
Publication #:	REL 2016118 General Ordering Information
Center/Program:	REL
Associated Centers:	NCEE
Authors:	David Stuit: Basis Policy Research; Mindee O’Cummings, Heather Norbury, Jessica Heppen, Sonica Dhillon, Jim Lindsay, and Bo Zhu: American Institutes for Research In collaboration with the Midwest Dropout Prevention Research Alliance
Type of Product:	Making Connections
Keywords:	Completion Rates (high school) Graduation Rates, High School High School States Students • at-risk
Questions:	For questions about the content of this Making Connections, please contact: Amy Johnson.