Inside IES Research

One year ago, IES introduced a new equity standard and associated recommendations to its Standards for Excellence in Education Research (SEER). The intent of this standard, as well as the other eight SEER standards, is to complement IES’s focus on rigorous evidence building with guidance and supports for practices that have the potential to make research transformational. The addition of equity to SEER is part of IES’s ongoing mission to improve academic achievement and access to educational opportunities for all learners (see IES Diversity Statement). IES is mindful, however, that to authentically and rigorously integrate equity into research, education researchers may need additional resources and tools. To that end, IES hosted a Technical Working Group (TWG) meeting of experts to gather input for IES’s consideration regarding the existing tools and resources that the education community could use as they implement the new SEER equity standard in their research, along with identifying any notable gaps where tools and resources are needed. A summary of the TWG panel discussion and recommendations is now available.

The TWG panel recommended several relevant resources and provided concrete suggestions for ways IES can support education researchers’ learning and growth, including training centers, coaching sessions, webinars, checklists, and new resource development, acknowledging that different researchers may need different kinds of supports. The meeting summary includes both a mix of recommendations for tools and resources, along with important considerations for researchers, including recommendations for best practices, as they try to embed equity in their research. 

The new SEER equity standard and accompanying recommendations have been integrated throughout the current FY 2024 Request for Applications. By underscoring the importance of equity, the research IES supports will both be rigorous and relevant to address the needs of all learners.   

This blog was written by NCER program officer Christina Chhin. If you have questions or feedback regarding the equity TWG, please contact Christina Chhin (Christina.Chhin@ed.gov) or Katina Stapleton (Katina.Stapleton@ed.gov), co-chair of the IES Diversity Council. If you have any questions or feedback regarding the equity standard or associated recommendations, please email NCEE.Feedback@ed.gov.

Until recently, limited data existed in education research focused on the LGBTQI+ community and their experiences. As this area of interest continues to grow, education researchers are learning how to effectively collect these data, interpret their implications, and use them to help improve the educational outcomes of LGBTQI+ identifying students. In this blog post, we review current federal recommendations for data collection and encourage researchers to submit FY 2024 applications focused on the educational experiences and outcomes of Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex (LGBTQI+) identifying students.

Collecting Data on Sexual Orientation and Gender Identities

In January 2023, the Office of the Chief Statistician of the United States released a report with recommendations on how to effectively design federal statistics surveys to account for sexual orientation and gender identities (SOGI). While this report is for a federal audience, the recommendations are relevant and useful for education researchers who wish to measure the identities and experiences of those in the LGBTQI+ community. Some suggestions include—

• Provide multiple options for sexual orientation identification (for example, gay/lesbian, straight, bisexual, use other term)
• Provide a two-question set in order to measure gender identity—one asking for sex assigned at birth, and one for current self-identification
• Provide write-in response and multiple-response options for SOGI-related questions
• Allow respondents to proceed through the survey if they choose not to answer unless answers to any of these items are critical for data collection

Education researchers looking to incorporate SOGI data into their studies can also use existing SOGI data collected by the National Center for Education Statistics (NCES) to support their research. A new NCES blog outlines the studies that collect SOGI information and outlines some initial findings from that data.

Funding Opportunities for Research to Improve Outcomes of LGBTQI+ students

In alignment with the SEER Equity Standard, IES encourages researchers to submit applications to the FY 2024 research grant competitions that support the academic and social behavioral outcomes of students who identify as LGBTQI+. IES is especially interested in research proposals that involve—

• Describing the educational experiences and outcomes of LGBTQI+ students
• Creating safe and inclusive learning environments that support the needs of all LGBTQI+ students.
• Identifying promising practices for school-based health services and supports, especially mental health services, that are accessible to and supportive of LGBTQI+ students
• Identifying systems-level approaches that reduce barriers to accessing and participating in high quality learning environments for LGBTQI+ students

Check out our funding opportunities page for more information about our FY 2024 requests for applications. If you have specific questions about the appropriateness of your research for a specific FY 2024 research competition, please contact the relevant program officer listed in the request for applications.

This blog is part of a 3-part Inside IES Research blog series on sexual orientation and gender identity in education research in observance of Pride month. The other posts discuss the feedback from the IES LGBTQI+ Listening and Learning session and the first ever learning game featuring a canonically nonbinary character.

This blog was produced by Virtual Student Federal Service intern Audrey Im with feedback from IES program officers Katina Stapleton (NCER - Katina.Stapleton@ed.gov) and Katherine Taylor (NCSER - Katherine.Taylor@ed.gov) and NCES project officers Elise Christopher (Elise.Christopher@ed.gov) and Maura Spiegelman (Maura.Spiegelman@ed.gov).

April is National Bilingual/Multilingual Learner Advocacy Month! In this guest blog, Dr. Ryan Williams, principal researcher at the American Institutes for Research, describes his IES-funded project focused on identifying factors that help explain variation in the effects programs have on English learner student outcomes using a broad systematic review and meta-analysis.  

Over the past two decades, empirical research on programs that support English language and multilingual learners has surged. Many of the programs that researchers have studied are designed to support English literacy development and are tailored to the unique needs of English learners. Other programs are more general, but researchers often study program impacts on English learners in addition to impacts on a broader population of students. Relatively few attempts have been made to identify common findings across this literature. Even fewer attempts have been made to identify meaningful sources of variation that drive program impacts for English learner students—that is, understanding what works, for whom, and under what conditions. To help provide educators and policymakers answers to those important questions, we conducted a systematic review and meta-analysis of the effectiveness of programs and strategies that may support English language learner students.

Our Systematic Review Process

We conducted a broad search that combed through electronic databases, unpublished ‘grey’ literature (for example, working papers, conference presentations, or research briefs), and sources that required hand-searching such as organizational websites. After documenting our primary decision-making factors within a review protocol, we applied a set of rigorous criteria to select studies for inclusion in the meta-analysis. We ultimately identified 83 studies that met our inclusion criteria. Each of these were randomized field studies that included English learner students in grades PK-12 and student academic learning outcomes such as English literacy, mathematics, science, and social studies. Each of the included studies was systematically coded to capture characteristics about the research methods, students and schools, settings, programs, outcome measures, and importantly, the program impacts that the studies reported. We then conducted a meta-analysis to understand the relationships between the characteristics we coded and the program impacts.

Preliminary Findings

We are still working on finalizing our analyses; however, our initial analyses revealed several interesting findings.

• Programs that included support for students to develop their first language skills tended to have larger improvements in student learning. This is consistent with prior research that suggests that supporting first language development can lead to improved learning in core content areas. However, the initial findings from this meta-analysis build on the prior research by providing empirical evidence across a large number of rigorous studies.
• There are some particularly promising practices for educators serving English learner students. These promising practices include the use of content differentiation, the use of translation in a student’s first language, and a focus on writing. Content differentiation aligns with best practices for teaching English learners, which emphasize the importance of providing instruction that is tailored to language proficiency levels and academic needs. The use of first language translation can be helpful for English learner students, as it can support their ability to access and comprehend academic content while they are still building their English proficiency. Focusing on writing can also be particularly important for English learners, as writing is often the last domain of language proficiency for students to develop. Our preliminary findings that English learner writing skills are responsive when targeted by instructional programs may hold implications for how to focus support for students who are nearing but not yet reaching English proficiency.
• The type of test used to measure program impact was related to the size of the program impact on student learning that studies found. Specifically, we found that it is reasonable to expect smaller program impacts when examining state standardized tests and larger impacts for other types of tests. This is consistent with findings from prior meta-analyses based on more general student populations, and it demonstrates the same applies when studying program impacts for English learner students. Statewide standardized tests are typically designed to cover a broad range of state content standards and thus may not reflect improvements in more specific areas of student learning targeted by a given program. On the other hand, researcher-developed tests may align too closely with a program and may not reflect broader, policy-relevant, changes in learning. Our initial evidence suggests that to understand program impacts for English learner students—or any group of students—we may want to use established, validated assessments but not only consider statewide standardized tests.

Next Steps

In terms of next steps, we will complete the meta-analysis work this summer and focus on disseminating the findings through multiple avenues, including a journal publication, review summaries on the AIR website, and future conference proceedings. In addition, we are working to deepen our understanding of the relationships identified in this study and explore promising avenues for practice and future research.

If you’d like to continue learning and see the results of this study, please continue to check back at AIR’s Methods of Synthesis and Integration Center project page, located here.

This blog was produced by Helyn Kim (Helyn.Kim@ed.gov), program officer for the English Learners portfolio, NCER.

When it comes to using research to inform practice, teachers tend to want succinct tips and strategies that can work in their own classrooms. Researchers can use social media channels to tailor their messages from their research findings and disseminate where teachers are already active. In this guest blog, Dr. Sam Van Horne, University of Delaware, describes the work that researchers conducted as part of the originally IES-funded Center for Research Use in Education. The goal of the center was to understand the gaps between researcher and practitioner perspectives on the usefulness of research in practice so that the center can address issues around how researcher communicate about their research, how practitioners can use research more effectively in their classrooms, and how to build stronger connections between the two communities.

Using a large cross-sectional survey of school-based practitioners, we found that practitioners report consuming research through multiple channels, and more than half of reported using social media in the last year with the goal of improving their practice. Social media channels, therefore, provide education researchers with an opportunity to connect with practitioners, but where are researchers likely to find teachers on social media? And how can researchers distill their findings for sharing in mediums that are vastly different than traditional academic forms? Here are some recommendations based on our research.

• Finding and Connecting with Educators on Social Media: One review of research about social media use among teachers found that Facebook and Twitter are some of the main sites that teachers use. But teachers also use Pinterest and Instagram as methods for learning from other teachers about teaching strategies. Posting in multiple channels may make it more likely that a message can reach educators. To find educators, researchers can search for public lists on education-focused topics or see who is using hashtags like #edtwitter, #TeachersofInstagram, or #EduTooters. By following lists, researchers can efficiently find educators to follow and tag (i.e., add the educator’s username to a social media message) with their messages about research-informed practice. This can aid with directly engaging practitioners and beginning conversations about applying research to practice.
• Using Hashtags or Tagging Specific People on Social Media: Social media networks like Twitter can overwhelm users with the volume of content being shared, so it’s critical to use tools like hashtags to find a practice-focused community who may be interested in applying research findings. Users search for content with hashtags that are discipline specific or likely to reach educators, such as #edutooters on Mastodon, #edutwitter on Twitter, or #teachersofinstagram or #teachersfollowteachers on Instagram. The key is identifying teachers or knowledge brokers (i.e., people or organizations who support practitioners in applying research evidence to teaching practice) that may be interested in the message and who may retweet or boost the message to their own followers.
• Tailoring Messages to Focus on What Practitioners Can Do: When the audience is identified, researchers can ask themselves, “What do I want this group to consider doing based on these research findings?” Then, social media messages can incorporate those ideas rather than just summarizing research findings. Social media messages describing how research can inform education practice should be economical and capture interest. Links to the original paper can be appended to a post for those who want to read more.
•  When possible, include links to publications or resources in publicly available repositories and not to versions in subscription-based journals. IES grantees can increase the visibility of their research by submitting their publications as grantee submissions in ERIC. This not only fulfills public-access requirements but also gives practitioners access to important information for improving teaching practice.  
• Incorporating Visual Elements to Attract Attention to Education Research Findings: Messages that incorporate visual elements or video are better suited for sharing on social media. The visual abstract is a succinct summary of research findings that is well-suited for sharing in social media platforms, and researchers have found that visual abstracts are more often shared on social media platforms than plain text about research. You can find guidance on creating visual abstracts here, though the authors suggest collaborating with a designer. These visual abstracts are suited for visual platforms like Pinterest or Instagram. Some journals make a regular practice of posting brief video messages from authors who explain their research study and the significance of the findings. Animations can also attract more attention to messages about research.

Disseminating education research on social media is not a “one-and-done” practice but should be part of a professional social media presence. Many guides exist for developing a professional social media presence, such as these for Twitter and LinkedIn. In addition to posting about research and its implications for practice, researchers can post about research or general issues in the field. This helps with building a following that will be more likely to see posts about research. There are other benefits to disseminating research on social media channels, including providing researchers with metrics about how many times their messages are shared or retweeted (or boosted, on Mastodon), as well as enabling research about optimal ways to share research to reach the broadest audience. In fact, Dr. Farley-Ripple, a leader of CRUE, and colleagues have received funding from the National Science Foundation for a research study to investigate the effectiveness of different dissemination strategies on social media, including the effectiveness of the translational visual abstract.

Connecting with educators on social media is a process. Researchers can begin by creating a presence on social media networks where educators are found and then post often about education and use hashtags to help make messages visible to educators. Messages can be succinct posts that include recommendations or strategies that are feasible for educators to adopt or include multimedia messages like the translational visual abstract to attract attention in a medium that is suited to visuals. Over time, it’s possible to learn what works and what doesn’t and adapt strategies for reaching educators, while keeping in mind that the tools and networks available now will undoubtedly adapt and change themselves.

Sam Van Horne is a Data Scientist at the Center for Research Use in Education and the Center for Research in Education and Social Policy at the University of Delaware.

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.

February is Career and Technical Education (CTE) month! As part of our 20^th anniversary celebration, we want to highlight the great work our CTE Research Network (CTERN) continues to accomplish. The blog below highlights NCER’s conversation with the Equity Working Group of the IES-funded CTE Research Network. 

The Equity Working Group (EWG) of the CTE Research Network (CTERN) has published a new resource for researchers on using an equity lens in developing and conducting CTE research: The Equity Framework for CTE Research. CTERN is hosting a free webinar on February 21st at 3:00 Eastern to provide an overview of the framework and how people can use it. In this blog, members of the Equity Working Group answered questions about the framework and why it is important. 

The framework has a focus on equity, but equity can mean different things to different people. How does the EWG define equity in this framework?

We strongly believe that every student should have the opportunity to engage in quality educational experiences. Students who are interested should have access to CTE programs, regardless of their background characteristics. And school systems should invest in students so that they can succeed in these programs. Ultimately, we find ourselves quoting the Wisconsin Department of Public Instruction’s definition because it neatly captures our position: “Every student has access to the educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background, and/or family income.”

Why did the EWG members believe that there was a need for an equity framework for CTE research?

CTE has a long and complicated history, including extensive tracking under its previous incarnation as vocational education. The CTE Equity Working Group was very conscious of this history and wanted to take steps to help ensure that CTE research was helping to ameliorate current inequities. As we say in the framework, “We believe that infusing equity throughout our research is critical to ensuring that research can make a difference in promoting equitable learning experiences and outcomes for all students who participate in CTE.”

We also recognized that many researchers (including ourselves) want to use an equity lens to do their research but lack practical guidance in what that looks like. The working group believed that a framework with concrete examples and tips would help CTE researchers have a clearer picture of what to do and would provide a tool for helping them think differently about their work.

How did the EWG create the framework?

This was a collaborative process that grew out of our first CTE Research Network meeting in 2018 or 2019. A group of us realized that incorporating an equity lens into our work would help us better answer questions that matter to communities. We decided to form a working group, which ended up including around 20 or so researchers, practitioners, and policy staff. We read a lot of good frameworks from different organizations on improving our research practices, so we decided to invest our energy in seeing how it may be applied to a CTE context.

How is the framework structured and what are some key takeaways?

It is important to note what this framework is and is not. This framework is not intended as a methodological primer or a replication of existing research guidance; it is intended to encourage researchers to think about their own work through an equity lens.

The framework starts with a brief history of equity in CTE, a description of the process of creating the framework, a list of vocabulary (we believe having a common language is critical), and a statement of the values that underlie the framework.

The rest of the framework is then organized by six stages of research: 1) project management; 2) research design, 3) measurement and data collection, 4) data analysis, 5) cost and resource equity, and 6) reporting and dissemination. In each section, we include a description of how to implement the stage with an equity-focused lens, with questions for researchers to consider and potential barriers. Throughout, we have included examples from current and future CTE research. We are looking for more examples, so people should feel free to reach out to us at jedmunds@serve.org to share how they are doing this work.

In creating summary products to go along with the framework, we identified six themes that cut across the different stages: ensure transparency, involve the community, develop diverse teams, take a systems approach, acknowledge and attend to bias, and demonstrate respect. These themes are summarized in an infographic.

How do you hope that people will use the framework?

We hope this will help start or further conversations among CTE researchers. We structured the framework around each stage of the research process, so anyone engaging in this work can find elements to incorporate or questions to consider individually and as a team, regardless of where they are in their work right now. For studies just getting off the ground, we did our best to illustrate how researchers can build an equity approach from the start of a project through its completion.

What are some examples of how the framework changed individual EWG members’ research practices?

Julie A. Edmunds (co-facilitator): Working on the framework has crystallized three high-impact equity-focused practices that I now try to infuse throughout my work. First, I pay much more attention to the role of systems in inequities. I try to look at upstream factors that might be causing disparities in educational outcomes as opposed to just documenting gaps that might exist between sub-groups. Second, when presenting those gaps (which we still do because it is useful information), I am much more conscious about how those gaps are displayed. For example, we focus on making sure that “White” is not considered the default category against which all others are compared. Third, we are creating processes to ensure that we share our findings with people who gave us the data. For example, we are sending practitioner-friendly products (such as briefs or infographics) to the school staff we interviewed whose insights formed the basis for some of our findings.

John Sludden (member): The framework has helped us think about our internal processes and keeps us focused on our audience, who we’re doing this for. I’m an analyst on the project, and I’ve been empowered to ask questions, conduct analyses, and present to our research partners at the New York City Department of Education. We’re currently thinking about ways to communicate findings to different audiences. At the moment, we’re working on a plan to share findings with principals of CTE high schools in New York City. Organizationally, we are also working on ways to directly engage students in the city, who know more about the system than we ever will. Similar to Julie, analytically, we have spent a lot of our time and attention on looking at the conditions under which students have not been well-served by the system, and ways that students may be better served by CTE.

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.