Skip Navigation
Title: Comparing Impact Findings from Design-Based and Model-Based Methods: An Empirical Investigation
Description: This report compares empirical results from different approaches to analyzing data from randomized controlled trials (RCTs). It focuses on how impact estimates compare between recently-developed design-based methods and traditional model-based methods. Design-based methods use the potential outcomes framework and known features of study designs to connect statistical methods to the building blocks of causal inference. They differ from model-based methods that have commonly been used in education research, including hierarchical linear model (HLM) methods and robust cluster standard error (RCSE) methods for clustered designs. This study re-analyzes nine past RCTs in the education area using both design- and model-based methods. The study finds that model-based and design-based methods yield very similar impact estimates and levels of statistical significance, especially when the underlying analytic assumptions (e.g., weights used to aggregate clusters and blocks) are aligned.
Online Availability:
Cover Date: July 2017
Web Release: July 25, 2017
Print Release:
Publication #: NCEE 20174026
General Ordering Information
Center/Program: NCEE
Type of Product: Research Report
Questions: For questions about the content of this Research Report, please contact:
Amy Johnson.