In order to better prepare for the Base Year (BY) 6th Grade data collection, the Item Validation Field Test (IVFT) will be conducted in winter 2016 and the Operational Field Test (OFT) in winter 2017. Separating the two field tests by a year gives us the opportunity to apply any lessons learned about recruitment and administration in the IVFT to refinements for the OFT. In this way, the OFT can be a more effective dress rehearsal for the main study. Then we will draw the national sample in fall 2017 and conduct Base Year data collection in winter 2018. The meaning of “MGLS:2017” will now center on the notion of the sample rather than on the initial year of testing.
On April 2, 2015 the Disabilities Expert Panel met to follow up on discussions that began in the winter at the Middle Grades Experts Meeting. Participating experts were Jose Blackorby, SRI International, Lindsey Jones, National Center for Learning Disabilities (NCLD), Jim Wendorf, NCLD, and Margaret McLaughlin, University of Maryland. Participants from other Department of Education offices were Jackie Buckley and Kim Sprague, National Center for Special Education Research (NCSER), and Richelle Davis of the Office of Special Education Programs (OSEP). We discussed sampling approaches that would impact the research questions that could be answered with MGLS data. The research questions that MGLS data will be able to answer are going to be about students with disabilities as they are found in general education schools in the U.S. (as opposed to all students with disabilities in the U.S.) Among other topics and questions: How can MGLS be as inclusive as possible? Are the three disability categories we have selected for oversampling the best choices? What accommodations can/should we offer? How are students with IEPs doing in the prevalent middle school models?
The Middle Grades Experts Meeting was convened in January at the NCES main office in Washington DC and brought together leaders from various organizations to begin building bridges to the research community who will be the ultimate users of the MGLS dataset. In attendance were Nancy Flowers, University of Illinois; Deborah Kasak, National Forum to Accelerate Middle-Grades Reform; Doug MacIver, Johns Hopkins University; Margaret McLaughlin, University of Maryland; Steve Mertens, Illinois State University; Karen Swanson, Mercer University. The goals of the meeting were introductory in nature and prepared the way for collaboration in the coming four years as the field tests and national data collections get underway. We hope to get ideas and community support for gaining the cooperation of schools and districts as we begin recruitment of both general education students and the special education students in our disability sample. Click here (5.17 MB) to get the background documents that were provided in the meeting packet.
In order to facilitate the best recruitment and field test outcomes possible, it was decided to make two changes to our plan. 1) There will be two types of field test, one for item validation and another to test operational aspects of recruitment and the data collection. 2) The Item Validation Field Test (IVFT) and the Operational Field Test (OFT) will take place from January-March 2016. The Winter 2016 field tests will be followed in Winter 2017 by the 6th grade base year round of data collection. Winter is considered a better testing window than spring as it helps us to avoid overburdening schools who participate in state testing.
We presented to the NCES Private Schools meeting in October and hope to build goodwill and recognizability with that sector of our general sample.
Cognitive laboratories took place with middle grades students to identify any issues with items proposed for the field test MGLS mathematics, reading and executive function measures. We also carried out cog labs for the parent, mathematics teacher and school administrator questionnaires. Focus groups for middle grades mathematics teachers helped us to understand the way to improve items and the questionnaire as a whole for that group. We also conducted a focus group of middle grades students on their use of technology terms so that our questionnaire items might be more responsive to the latest trends. Students in the three IDEA categories (autism, emotional disturbance, specific learning disability) that we have selected for special focus in MGLS:2017 were also interviewed and assessed for their understanding of items and ability to complete assessments and questionnaires. Preparations for the spring 2015 field test are underway, specifically school recruitment.
A MGLS:2017 Reading Content Review Panel took place on April 14th and included John Sabatini of the Educational Testing Service (ETS) who is the lead on the new reading literacy assessment GISA. Members of his team Tenaha O'Reilly and June A Zumoff were also in attendance as were additional expert panelists Donna Alvermann of the University of Georgia, Joseph Magliano of Northern Illinois University, Sheryl, Lazarus of the National Center on Educational Outcomes (NCEO).
A Disabilities Content Review Panel took place on April 29th to discuss e.g. sampling and selection of IEP students to sample specifically and also to advise on the design of instruments and accommodations for all students with disabilities who occur in the survey either naturally or through sampling at higher rates. Expert panelists were Jose Blackorby of SRI International, Diane Bryant of the University of Texas, Austin, Lynn Fuchs of Vanderbilt University, Sheryl Lazarus of the National Center on Educational Outcomes (NCEO), Martha Thurlow of the National Center on Educational Outcomes (NCEO), and Mitchell Yell of the University of South Carolina.
We have narrowed our special sample choices for disabled students to three of the IDEA categories as follows: 1) specific learning disabilities, 2) emotionally disturbance, and 3) autism. While all students who can take a test using accommodations will be included in our study, the three selected groups will be sampled at a higher rate to assure that researchers have the statistical power needed for meaningful analysis. This is a pilot effort--NCES longitudinal studies has never before sampled specifically for IEP students and felt better results may be obtained by choosing fewer categories for this first attempt.
An exciting development is in the area of inclusion. We will draw a special sample of disabled students in several of the IDEA categories as follows: 1) specific learning disabilities, 2) ADD/ADHD children under the “Other Health Impairment” category, 3) speech or language impairment, 4) emotionally disturbed, and 5) autism. While all students who can take a test using accommodations will be included in our study, these selected groups will be augmented to assure that researchers have the statistical power needed for meaningful analysis.
Because of the special sample of students with disabilities and the notion that mathematics performance may be moderated by reading level, we are adding a measure of reading proficiency to the academic test battery based on work done in IES’s Reading for Understanding grant program. Details forthcoming!
We completed a series of Content Review Panels to determine the design of the instruments to be field tested in Spring 2015. The Mathematics panel included Kathy Heid of Pennsylvania State University, Margie Hill of the University of Kansas, Lisa Keller, University of Massachusetts, Amherst, Tom Loveless of the Brookings Institution, Ed Nolan of Montgomery County Public Schools, Maryland, Linda Wilson, and Paul Sally of the University of Chicago. Our panel on Executive Function included James Byrnes of Temple University, Lisa Jacobson of the Kennedy Krieger Institute and Johns Hopkins University School of Medicine, and Dan Romer of the University of Pennsylvania. Socioemotional instruments and teacher and family surveys were advised by James Byrnes of Temple University, Susan Dauber of Blue Stocking Research , Scott Gest of Pennsylvania State University, Richard Lerner and Tama Leventhal both of Tufts University, and Russell Rumberger of the University of California, Santa Barbara. Our School Administrator questionnaire received input from Susan Dauber of Blue Stocking Research, George Farkas of the University of California at Irvine, Jeremy Finn of the State University of New York at Buffalo, Tom Loveless of the Brookings Institution, and Ed Nolan of Montgomery County Public Schools, Maryland.