Variability in Pretest-Posttest Correlation Coefficients by Student Achievement Level

Title:	Variability in Pretest-Posttest Correlation Coefficients by Student Achievement Level
Description:	State assessments are increasingly used as outcome measures for education evaluations. The scaling of state assessments produces variability in measurement error, with the conditional standard error of measurement increasing as average student ability moves toward the tails of the achievement distribution. This report examines the variability in pretest-posttest correlation coefficients of state assessment data for samples of low-performing, average-performing, and proficient students to illustrate how sample characteristics (including the measurement error of observed scores) affect pretest-posttest correlation coefficients. As an application, this report highlights how statistical power can be attenuated when correlation coefficients vary according to sample characteristics. Achievement data from four states and two large districts in both English/Language Arts and Mathematics for three recent years are examined. The results confirm that pretest-posttest correlation coefficients are smaller for samples of low performers, reducing statistical power for impact studies. Substantial variation across state assessments was also found. These findings suggest that it may be useful to assess the pretest-posttest correlation coefficients of state assessments for an intervention’s target population during the planning phase of a study.
Online Availability:	Download, view and print the report as a pdf file. (1.3MB)
Cover Date:	September 2011
Web Release:	September 7, 2011
Publication #:	NCEE 20114033
Center/Program:	NCEE
Authors:	Russell Cole, Joshua Haimson, and Irma Perez-Johnson: Mathematica Policy Research. Henry May, University of Pennsylvania Consortium for Policy Research in Education
Type of Product:	Evaluation Report
Keywords:	Achievement (Student) Assessment Measurement Error Studies States • state-level assessments
Questions:	For questions about the content of this Evaluation Report, please contact: Erin Pollard.