IES Blog

In 2014, IES funded one of two R&D Centers on Knowledge Utilization (or Knowledge/Evidence Use) to explore how 1) education researchers can make their work more relevant and useful to practitioners located in state and local education agencies and in individual schools, 2) the work of practitioners can inform research efforts, and 3) practitioners can make decisions based on research evidence. The National Center for Research in Policy and Practice (NCRPP) at the University of Colorado at Boulder recently completed its grant. Corinne Alfeld, a Program Officer in IES NCER, talked with Principal Investigator Bill Penuel about the Center’s findings and recommendations. The full list of Center staff and collaborators can be found on NCRPP’s website.

What were the outcomes of your IES-funded KU R&D Center? 

Our center studied how school and district leaders accessed and used research through a nationally representative survey and through case studies of school district decision making, research-practice partnerships, and a professional association of state leaders in science. Across all studies, we found that leaders highly valued research. At the same time, we found some things that might be surprising to many researchers:

• Most leaders accessed research through their relationships and networks rather than through web sites, journal articles, or resources like the What Works Clearinghouse. The most common ways leaders accessed research was through their own professional associations, conferences, and colleagues in education settings. In some cases, these networks provided leaders with access to high-quality research. In our study of the professional association of science leaders, leaders cited National Academies of Sciences, Engineering, and Medicine consensus study reports as the most shared and used among members.
• Leaders used research for a variety of purposes and for a range of decision-making activities not only to make decisions about what programs to adopt. Many leaders, for example, design professional development for educators for which there may be no program to adopt. In our case studies of district research use, we found that leaders did rely on research—conceptually—to inform their design activities. Research use was embedded within the ongoing decision-making routines for designing, implementing, and evaluating professional development activities.
• Leaders did not turn to impact studies of individual interventions when they looked for research. Instead, they more commonly turned to books and other kinds of publications that provided syntheses or summaries of research. A common thread was that these sources of research provided frameworks for action, broken down into clear steps they could follow.

How can researchers use your findings to improve their dissemination efforts?

Despite the value leaders placed on research, there is clearly room for improvement with respect to dissemination. More than half of respondents to our survey said that by the time research was published, it was no longer valuable to them. Here are two strategies that researchers might find useful.

1. Long-term research-practice partnerships (RPPs) were sites where leaders found research to be both timely and relevant to them. These partnerships come in different shapes and sizes and have different goals. However, all engage educators in helping define the very questions that will be addressed in the research, and some also engage in co-design and testing of interventions to address persistent problems of practice and to work toward visions of more equitable systems of education.
2. Embedding researchers within leaders’ professional organizations can help disseminate research in a timely manner. These members present regularly at association meetings and conferences. They also participate in committees, where they develop tools to inform ongoing leadership activities.

What these strategies have in common is that dissemination is not an afterthought. In fact, dissemination is not a good word for what these researchers are doing. A better word is engagement. Researchers are engaging educators throughout the process of research and development, not just at the end.

How can your findings be used to improve practitioner access to and use or consideration of research findings?

Many of the strategies for engagement involve educators engaging with various aspects of research directly. Years ago, Weiss and colleagues described this as its own form of research use, called process use. Engaging educators in the actual research process does something that is important for supporting research use, namely giving them time to make sense of research and its implications for their work.

We found evidence that involvement in RPPs for educators was helpful to their own policymaking and practice. More than three-quarters, for example, said that their external partners shaped the design of professional development, and many also said that their partnership helped to integrate newly developed practices in the partnership.

What are your plans for future work in this area?

At present, NCRPP is involved in two exciting new projects. The first is a project funded by the William T. Grant Foundation, which focuses on developing and validating measures to assess the effectiveness of RPPs. We’re using a framework developed by Erin Henrick and colleagues to evaluate RPPs, and we’ve gathered survey and interview data from more than 60 RPPs. Our goal is to develop formative measures to help RPPs evaluate progress on each of the five dimensions of the framework.

The second project is funded by the Wallace Foundation to study and support equity-centered leadership and districts in forming partnerships with researchers as they develop and test strategies for creating equity-centered leadership pipelines. Both projects are being undertaken in collaboration with the National Network of Education Research-Practice Partnerships.

What do you see as the next steps for this field?

While it is tempting to suggest “more research is needed,” what is needed is an “evidence-informed” approach to evidence use—the application of what we already know about evidence use when it comes to policy and practice. That requires us to shift focus away from imagining that better, plain-language research briefs will help us improve research use. Instead, we need to encourage researchers to engage in more substantive ways with practice throughout the research process, to improve its relevance and timeliness.

We also need to embrace a broader conception of the kinds of evidence and information that can inform decision making, one that reflects the range of information that leaders currently use and could turn to. Of particular importance is considering the experiences of those students, families, and communities to whom we owe a great education debt as important sources for decision making. If we take a broader view of evidence, a new question emerges: How can we consider and integrate different sources of evidence in a way that is informed by values such as equity and justice into decision making? This is the sort of question I hope the field can pursue in the future.

Findings from the 2015 IES-funded Center for Research Use in Education (CRUE) at the University of Delaware will be highlighted in a blog in 2022. Stay tuned! If you have further questions, please contact Corinne.Alfeld@ed.gov.

IES values and encourage collaborations between researchers and practitioners to ensure that research findings are relevant, accessible, feasible, and useful. In 2017, Dr. Wendy Reinke, University of Missouri, received IES funding to formalize the Boone County School’s Mental Health Coalition by strengthening their partnership and validating the Early Identification System (EIS) to screen for social, emotional, behavioral, and academic risk among K-12 students in rural schools. Building on these successes, Dr. Reinke now leads the National Center for Rural School Mental Health (NCRSMH), a consortium of researchers leading efforts to advance mental health screening and support in rural communities.

Bennett Lunn, a Truman-Albright Fellow at IES, asked Dr. Reinke about the work of the original partnership and how it has informed her efforts to build new partnerships with other rural schools around the country. Below are her responses.

What was the purpose of the Boone County Schools Mental Health Coalition and what inspired you to do this work?

In 2015, our county passed an ordinance in which a small percent of our sales tax is set aside to support youth mental health in our community. As a result, the schools had visits from many of the local mental health agencies to set up services in school buildings. The superintendents quickly realized that it would be wise to have a more coordinated effort across school districts. They formed a coalition and partnered with researchers at the University of Missouri to develop a comprehensive model to prevent and intervene in youth mental health problems. The enthusiasm of our school partners and their willingness to consider research evidence to inform the model was so energizing! We were able to build a multi-tiered prevention and intervention framework that uses universal screening data to inform supports. In addition, we were awarded an IES partnership grant to help validate the screener, conduct focus groups and surveys of stakeholders to understand the feasibility and social validity of the model, and determine how fidelity to the model is related to student outcomes. The EIS is now being used in 54 school buildings across six school districts as part of their daily practice. 

Were there advantages to operating in a partnership to validate the screener?  

The main benefit of working in partnership with school personnel is that you learn what works under what circumstances from those directly involved in supporting students. We meet every month with the superintendents and other school personnel to ensure that if things are not working, we can find solutions before the problems become too big. We vote on any processes or procedure that were seen as needing to change. The meeting includes school personnel sharing the types of activities they are doing in their buildings so that others can replicate those best practices, and we meet with students to get their perspectives on what is working. In addition, the university faculty bring calls for external funding of research to the group to get ideas for what types of research would be appropriate and beneficial to the group. Schools are constantly changing and encountering new challenges. Being close to those who are working in the buildings allows for us to work together in forming and implementing feasible solutions over time.

What advice do you have for researchers trying to make research useful and accessible to practitioners? 

Be collaborative and authentic. Demonstrate that you are truly there to create meaningful and important changes that will benefit students. Show that your priority is improving outcomes for schools and students and not simply collecting data for a study. These actions are vital to building trust in a partnership. By sharing the process of reviewing data, researchers can show how the research is directly impacting schools, and practitioners have an opportunity to share how their experience relates to the data. A good way to do this is by presenting with practitioners at conferences or collaboratively writing manuscripts for peer reviewed journals. For example, we wrote a manuscript (currently under review) with one of our school counselor partners describing how he used EIS data in practice. Through collaboration like this, we find that the purpose and process of research becomes less mysterious, and schools can more easily identify and use practices that are shown to work. In this way, long-term collaboration between partners can ultimately benefit students!

How does the work of the original partnership inform your current work with the National Center for Rural School Mental Health? 

We are bringing what we have learned both in how to be effective partners and to improve the model to the National Center for Rural School Mental Health. For instance, we are developing an intervention hub on our Rural Center website that will allow schools to directly link evidence-based interventions to the data. We learned that having readily available ideas for intervening using the data is an important aspect of success. We have also learned that schools with problem solving teams can implement the model with higher fidelity, so we are developing training modules for schools to learn how to use the data in problem solving teams. We will be taking the comprehensive model to prevent and intervene with youth mental health and using it in rural schools. We will continue to partner with our rural schools to continuously improve the work so that it is feasible, socially valid, and important to rural schools and youth in those schools.

Dr. Wendy Reinke is an Associate Vice Chancellor for Research at the University of Missouri College of Education. Her research focuses on school-based prevention interventions for children and youth with social emotional and behavioral challenges.

Written by Bennett Lunn (Bennett.lunn@ed.gov), Truman-Albright Fellow, National Center for Education Research and National Center for Special Education Research

Research on how people learn is critical for informing the design of effective education technology products. To design products that improve student learning, we need to understand how students approach solving problems, the information they need to adopt optimal solution strategies, the skills that underlie success in particular academic domains, the best ways to arrange information on a screen to guide student attention to relevant information, and the best study strategies for optimizing learning and retention. Through its research grants programs, IES has invested in research projects to develop and test education technology products based in the science of learning.

The 2021 ED Games Expo, which takes place virtually from June 1-5, features a number of these products, and you can learn more about them through a mix of live and pre-recorded sessions and videos. Most are ready to demo now. Students and educators can send questions about the research, game-play, or ed tech experience directly to the participating researchers. Here are just a few of the many ways you can interact with IES-funded researchers at the Expo:

1. The Virtual Learning Lab (VLL) will be hosting a live, virtual session on Friday, June 4^th from 4:00-5:30pm Eastern Time to celebrate their 5-year research collaboration to explore precision education in the context of algebra instruction. The VLL developed an AI-powered video recommendation system that personalizes math instruction within Math Nation. The researchers measured student ability and engagement, detected effects of virtual learning environment usage on achievement, and identified characteristics of effective online tutoring. The session will feature short talks and opportunities for Q&A: 

• A Video Recommendation System for Algebra (Walter Leite, University of Florida)
• Reinforcement Learning for Enhancing Collaborative Problem Solving (Guojing Zhou, University of Colorado Boulder)
• Natural Language Processing for VLE Research (Danielle McNamara, Arizona State University)
• Identifying Pedagogical Conversational Patterns in Online Algebra Learning (Jinnie Shin, University of Florida)
• Scaling Items and Persons for Obtaining Ability Estimates in VLEs (A. Corinne Huggins-Manley, University of Florida)
• Measuring Student Ability from Single- and Multiple-Attempt Practice Assessments in VLEs (Ziying Li, University of Florida)
• Detecting Careless Responding to Assessment Items in VLEs (Sanaz Nazari, University of Florida)
• Personalization, Content Exposure, and Fairness in Assessment (Daniel Katz, University of California, Santa Barbara)
• Fair AI in VLEs (Chenglu Li, University of Florida)

2. The ED Games Expo YouTube Playlist, which will be posted on June 1^st on the event page, features 26 products developed with IES grant funding. For ed tech products informed by research on how people learn, check out the products funded through the Cognition and Student Learning topic:

• Graspable Math allows math teachers to assign interactive algebra tasks and turns equations into tangible objects that middle school and high school students can manipulate to practice and explore. Teachers can follow live, step-by-step, student work.
• eBravo Boulder Reading Intervention is a self-paced personalized reading comprehension curriculum that teaches secondary students the problem-solving skills good readers use to learn from challenging texts, in this case in the science discipline of ecology. Reading strategies and exercises are guided by well-researched models of reading comprehension, helping students build deep, durable, and reusable knowledge from text. 
• iSTART and Writing Pal are interventions designed for middle school students, high school students, and young adults improve their reading and writing skills. Within these interventions, students play games to practice reading comprehension and writing strategies.
• All You Can Eat, Gwakkamole, and CrushStations are part of a suite of Executive Function skill-building games, designed to improve student shifting, inhibitory control, and working memory respectively.

We hope you can join us for this exciting event in June to learn more about and try out all the research-based products ready to be used in our nation’s schools. For more information on the featured resources and online events, please see this blog.

Written by Erin Higgins (Erin.Higgins@ed.gov), Program Officer for the Cognition and Student Learning program, National Center for Education Research

This May as we recognize National Mental Health Awareness Month, schools around the country are welcoming students and educators back for in-person instruction after more than a year of remote or hybrid teaching and learning. One issue schools must consider during this transition back is the increase in mental health concerns among adults, young adults, and adolescents this past year. Here at IES, we support research that explores, develops, and tests innovative, field-initiated approaches to support mental health in schools and classrooms. This new IES blog series will explore school-based mental health by looking at IES-funded research that helps answer the five Ws:

• Why school-based mental health? The first blog in the series will consider the benefits of school-based mental health such as providing increased access to services, especially for children of color, and potentially counteracting the stigma some associate with mental health treatment.

• What can schools do to support the mental health of their students and staff? The second blog in the series will highlight several projects that are developing innovative new ways to provide mental health services in school settings.

• When during the school day can schools implement these mental health practices so that they do not compete with the academic/instructional goals of school? The third blog in the series will highlight a variety of projects that delve into the implementation challenges inherent to providing school-based mental health services and support.

• Who in the school should implement these mental health practices? The fourth blog in the series will explore the critical scale up challenge for schools of having staff with adequate time who can be appropriately trained to provide mental health supports to students.

• Where can these mental health practices be implemented? The final blog in the series will investigate the implementation challenges of different education settings (PreK, elementary, middle, high school, postsecondary) for school-based mental health programs and practices.  

See these blogs for more information about some of the school-based mental health research supported through the two IES research centers, the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER).  

Written by Emily Doolittle (Emily.Doolittle@ed.gov), NCER Team Lead for Social Behavioral Research at IES

In 2018, Dr. Allison Master and co-PI Andrew Meltzoff were awarded a grant, Gender Stereotypes in STEM: Exploring Developmental Patterns for Prevention. This 4-year project explores how and when gender stereotypes about STEM career pathways emerge. The study also seeks to identify ways to mitigate the effects of such stereotypes, such as whether a growth mindset can lead to changes in student attitudes and outcomes toward STEM. As an undergraduate student majoring in microbiology at UCLA, Yuri Lin, virtual intern at NCER, was interested in learning more about gender inequalities and stereotypes in STEM education. She recently had a chance to talk with Dr. Master about her research and its implications for increasing STEM participation among women.

How is American culture affecting the STEM gender gap, and how does the US compare to other countries on this issue?

When children grow up in American culture, they see lots of TV shows and books where mathematicians, scientists, and engineers are men. STEM-based toys are also heavily marketed toward boys rather than girls. Some countries have begun changing the portrayal of gender stereotypes in the media. For example, the UK’s Advertising Standards Authority has recently started banning TV commercials that reinforce gender stereotypes. Some cross-national studies have shown that gender-STEM stereotypes favoring men are linked to women’s lower success and participation in STEM. The United States is one of many Western countries in which women have more equality and freedom to choose their careers but are much less likely to choose STEM careers than men. We still have a lot of work to do in the United States to break down barriers for women in STEM, and we need to focus on helping girls and women see the value in choosing pathways into STEM.

Why do you think it is important to examine growth mindset as a potential way to reduce the effects of stereotypes and increase STEM interest in students?

Growth mindsets are beliefs that personal characteristics can be changed, through effort or the right strategies. This is contrasted with fixed mindsets, which are beliefs that those characteristics can’t be changed. Growth mindsets are particularly helpful for struggling students. Students who have a growth mindset remain focused on learning rather than looking smart, believe effort is important, and stay resilient even when they experience setbacks. These attitudes translate into putting forth more effort and determination, which lead to greater success. In our project, we want to know if a growth mindset can help girls stay motivated in computer science, a subject that can have a steep learning curve. Girls in particular often get discouraged when they feel that they don’t have what it takes to succeed in STEM. We hope that teaching girls to have a growth mindset will protect them from these negative stereotypes and increase their confidence in themselves and their sense of belonging in computer science.

Considering that your project includes students from grades 1 to 12, how do you plan to share your findings with teachers, students, and policymakers? Are there differences in how you might communicate the information for different age groups?

As a developmental psychologist, I think it’s important to communicate the information about different age groups to everyone! It can be very valuable to frame student motivation in the broader context of how students are growing and changing. Students start to endorse stereotypes about computer science and engineering very early—Grades 1-3—so elementary school is a great time to start counteracting stereotypes by showing a broad representation of who enjoys and succeeds in STEM. We start to see big gender gaps in computer science interest during middle school, so this is a great time to have girls participate in fun and engaging coding classes. And we’ve already noted how important it is for girls in high school to have a growth mindset in their STEM classes.

We have different goals for communicating with teachers, parents, and policymakers. We know that teachers are very busy, so we try to condense things into the most important practical tips. We’ve made short videos and infographics about our research for teachers. For policymakers, we write policy briefs, which combines our research with other findings that are relevant to education policy. And when we talk to parents, we try to focus on the importance of the experiences they provide for their kids. We really value spreading the word about our research to make sure it reaches people who can use it to make a difference. For more information and access to the various resources, please visit the I AM Lab website.

Allison Master, PhD (@AllisonMaster), a developmental psychologist and an assistant professor at the University of Houston, has conducted extensive research on the development of motivation and identity in STEM education. 

Written by Yuri Lin (ylin010101@g.ucla.edu), intern for the Institute of Education Sciences and a Microbiology, Immunology, and Molecular Genetics major at UCLA.