Inside IES Research

Notes from NCER & NCSER

Leveraging Diversity of Academic Disciplines and Cultural Experiences to Advance Education Research

This year, Inside IES Research is publishing a series of blogs showcasing a diverse group of IES-funded education researchers and fellows that are making significant contributions to education research, policy, and practice. For Asian American and Pacific Islander Heritage Month, we asked researcher Mingyu Feng, a senior research associate at WestED, to discuss her career journey. Dr. Feng serves as principal investigator of the IES-funded ASSISTments and MathSpring efficacy studies, which examine the effects of intelligent tutoring systems, data-driven instruction, and formative assessment on student learning outcomes.

How did you become interested in a career in education research? How have your background and experiences shaped your scholarship and career?

When I first came from China to the United States to pursue a PhD in computer science, I wasn’t thinking of having a career in education research. My goal was to become a computer scientist who plays with algorithms and codes every day. Then, I was surprised to learn how many U.S. students trailed their peers academically in a highly developed country like America. The 2002 NAEP data indicated that only 30% of 8th graders were at or above the proficient level in reading or math. My first thought when I saw that statistic was, “There must be a way to help these students.”

I pursued a PhD in intelligent tutoring systems at Worcester Polytechnic Institute out of a desire to leverage technology to boost student learning. An intelligent tutoring system (ITS) is a computer system that aims to automatically provide immediate and customized instruction, feedback, or intervention to learners. Building ITSs requires a highly interdisciplinary field, where computer technology intersects with artificial intelligence, data mining, learning sciences, cognitive sciences, and education. As a graduate student, I developed systems and analyzed student learning data. I built upon my prior math knowledge from studying engineering and taught myself statistical modeling and learned about experimental design methods. I was also fortunate to be able to visit classrooms and work directly with educators and students to understand how critical it is for a student to receive needed support and for a computer system to be effectively integrated into an educator’s classroom routine. This experience inspired me to pursue a career in applied education research. Since then, my research career has focused on the development and research of education technologies and conducting rigorous evaluations of their impact on learning or practices in authentic education settings.

What has been the biggest challenge you have encountered, and how did you overcome the challenge?

As a first-generation immigrant, I haven’t directly experienced the U.S. K-12 and college education system and was initially less familiar with U.S. policies and practices. I asked a lot of “naïve” questions—What’s the difference between a public school and a charter school? How does a district decide the adoption of a supplemental program or a core curriculum? Does 5th grade belong to elementary school or middle school? Does a teacher or a school have discretion regarding instructional practices? It was a long and steep learning curve, but I found connecting directly with students, educators, administrators, and policymakers to be beneficial for learning about the education system. Listening to their needs, observing classrooms, and discussing research and findings in a meaningful way with practitioners provided me with the context and inspiration I needed as a researcher. When I saw an exhausted teacher running around to put out fires in the classroom, or a frustrated student staring at the computer screen, I knew there was still a long way for us edtech developers and researchers to go.

I also recognized that my cultural background and resulting perspective on education could be both a challenge and an asset. In many East and Southeast Asian cultures, Confucian ideals such as respect for elders, deferred gratification, and discipline are strong influences. Traditionally, Asian parents teach their children to value educational achievement, respect authority, feel responsibility for relatives, and show self-control. These perspectives were infused in my upbringing and influenced my approach to understanding education in the United States where diverse cultures thrive. I worked to gain perspective-taking skills to understand situations from other positions, to consider other beliefs, experiences, and viewpoints.

How can the broader education research community better support the careers and scholarship of researchers from underrepresented groups?

Early exposure to research and opportunities can be quite helpful. My doctoral training expanded my view of career options beyond academia and helped me see the value of applied research and the impact research can have on practices.

Recently, IES has advocated specifically for inclusion of emerging scholars from underrepresented groups in RFAs and during the reviewing process. That’s a great way to support scholars from these groups. Just for fun, I skimmed the list of 1,500 PIs of IES funded grants and found about 50 first or last names resembling Asian names. With acknowledgement of this less-than-rigorous approach, this very rough estimate of 3-4% suggests there are not a whole lot of IES PIs with Asian heritage. Therefore, I really appreciate the increased attention and encouragement IES has given to addressing underrepresentation in the education research community and would love to meet more scholars like me at the PI meetings.

In addition, I’d encourage project directors to think more creatively when considering institutional partnerships, building a staff team, or forming an advisory board. By including collaborators or advisors from underrepresented groups, the team benefits from the breadth of talent, perspectives, and skills that arise from diversity.

What advice would you give to emerging scholars from underrepresented, minoritized groups that are pursuing a career in education research?

My first piece of advice is to be confident and brave. Believe in yourself and in the value you bring to the table. Sometimes, this means stretching outside your comfort zone or persevering towards a goal you are passionate about. In Chinese culture, modesty is viewed as a virtue. I’ve always been told “modesty helps one go forward” and “silence is golden.” Yet, I’d encourage emerging scholars from minoritized groups to put their best selves forward and display their pride. 

My second piece of advice is to find someone you trust, a mentor or an advisor, who will show you how to navigate the field and provide guidance for your academic and career advancement. A great mentor can show you your strengths and weaknesses, encourage and advocate for you, and support your growth by creating opportunities and connecting you with collaborators. For someone from underrepresented groups, I found it is best to have someone who can speak up for you when you are not present in the room.


Mingyu Feng is a senior research associate with WestEd’s Learning and Technology team. She leads large-scale grants focused on leveraging education technologies to transform science and mathematics instruction to improve student learning.

Produced by Wai Chow (Wai-Ying.Chow@ed.gov), program officer for the Effective Instruction grant program within the National Center for Education Research.

Technical Working Group of Education Policy Leaders and Researchers Advises NCER on Education Policy Research Priorities and Strategies

The National Center for Education Research convened a virtual technical working group (TWG) of education policy leaders and researchers to discuss strategies for improving K-12 education systems and policy research funded by NCER. The official summary is now posted on the IES website. This blog post provides a snapshot of that summary.

 

What are the most pressing education systems and policy issues in need of research?

Prior to the meeting, TWG members were asked to identify what policy topics need evidence to inform their decision making, and they identified over 20 issues of pressing concern. The top-identified education policy issues, with at least 3 nominations across both policy leader and researcher groups, are listed below (with the number of nominations in parentheses).

  • Equitable access to high quality instruction (13)
  • Education technology and online instruction (11)
  • Equity in school and student funding (10)
  • Diversity of teachers and equity of pay (8)
  • Socio-emotional learning (SEL), relationships and student engagement (8)
  • Leader Professional Development (6)
  • Teacher professional development (5)
  • Better capacity for data analysis management, communication (5)
  • Early childhood education (5)
  • Career preparation/advising/transitions (5)
  • Culturally relevant pedagogy/anti-racism/anti-bias (3)
  • Special education/English Learners (3)
  • Covid-related learning loss (3)

The discussion during the meeting focused on a broad range of policy, research, and dissemination issues, which did not always align with the above issues. NCER staff organized the themes from the day’s discussion below to highlight where the TWG members expressed general consensus. More detail about each area can be found in the full TWG summary on the IES website.

  • Understanding and Addressing Inequity in Education Systems: The TWG members agreed that the most pressing issue is equity. The COVID-19 pandemic exacerbated and revealed systemic inequity, which affects students who are from racial/ethnic minority backgrounds, low-income households, or other marginalized groups.
  • Improving Use of and Access to Education Technology and High-Quality Online Instruction: With the heavy reliance on online instruction throughout the pandemic, COVID-19 has created new urgency for closing the digital divide. In addition to understanding how systems support or impede access to education technology, TWG members also noted a need for education research focused on questions of teaching and learning in remote and hybrid environments, including professional development and systems-level support for both teachers and students for using and engaging with technology.
  • Recruiting and Retaining a Diverse Teaching Workforce: TWG members noted that teacher shortages, especially in high poverty districts, have been a concern for years, and that there is a need for evidence on how to make access to high quality teachers more equitable. Researchers could examine how recruitment strategies can diversify the teacher workforce through strategies such as incentives specifically for teacher candidates of color and adopting culturally relevant practices.
  • Providing Access to Student and Educator Mental Health Supports: TWG members observed that COVID-19 has been stressful for students, parents, and educators. Remote learning appears to prevent many students from connections that support emotional and behavioral health and to increase the vulnerability of students with high-risk home environments. Research and research syntheses are needed to guide policymakers on what works best in allocating resources to meet student and educator mental health needs and how to connect with community resources and improve systems of support beyond the school.
  • Engaging and Re-Engaging Students: Chronic absenteeism is a major problem in many schools, exacerbated by the pandemic. TWG members pointed out that a key policy question for many LEAs and SEAs is how to re-engage these “missing” students. Researchers can help education leaders understand what evidence-based interventions are available to re-engage students and begin to address disparities in pandemic learning loss.
  • Preparing K-12 Students for Careers: TWG members agreed that researchers, policymakers, and educators should engage and collaborate more frequently with employers to inform what career-aligned experiences should be offered to students in school. Research could contribute to understanding how best to support local communities of practice that include schools, businesses, intermediaries, and community-based organizations with the shared goal of preparing students for careers.
  • Modernizing Assessments: TWG members agreed that education systems should be modernized to assess and address basic skills and learning needs quickly, such as with interim or formative assessments. Research is needed to understand how to use assessment for both accountability purposes as well as to support diagnosis and student progress monitoring. Additionally, research is needed to guide educators on authentic and performance-based assessments.
  • Improving Data-Driven Decision-Making in Schools: TWG members noted that education researchers could provide guidance on identifying a set of core variables, systematically collecting data and metrics, and building data sharing platforms and data agreements. TWG members pointed out that, in many cases, education agencies and educators do not necessarily need more data but more training to build capacity to analyze and use the data they have.
  • Examining Education Finance: TWG members noted that research can generate high-quality information to help policymakers understand how school systems could leverage financial resources to help the most underserved learners and communities. TWG participants noted that a key question for which additional evidence is needed is how much funding is needed to provide the breadth of services, including wraparound services, required to support learning in the poorest schools.
  • Creating Adaptive Education Systems: COVID-19 has shown that education systems must better prepare for emergencies, and TWG members worried that the pandemic will have a lasting effect on student achievement and attainment. The COVID-19 pandemic has provided the education and research community with an opportunity to learn from what went well, what did not, and to propose strategies to put in place to ensure rapid responses to future emergencies and moments of crisis.

How can NCER better support research on education systems/policy issues?

NCER staff asked for recommendations on improving its engagement with education systems and policy work. TWG member recommendations are organized according to five themes.

  • Support a Systems Approach to Systems and Policy Research: TWG members recommended that NCER encourage researchers to untangle and understand broader, dynamic education systems and processes, and to develop methods that capture and account for changing contexts. The TWG encouraged an interdisciplinary approach with different stakeholder perspectives, methods, and measures to move the field forward.
  • Encourage Partnerships with Key Stakeholders: TWG members felt that the relevance of research proposals could be increased with more collaboration between researchers and education leaders. Partnership between researchers and practitioners is one strategy for increasing the local relevance of research and its applicability to specific local questions.
  • Support Rapid Research to Practice Efforts: TWG members agreed that education policy research results should be disseminated to the field quickly. Rapid cycle evaluation methods, such as plan-do-study-act continuous quality improvement approaches can help to inform policy solutions, and while not nimble enough to support quick turnaround studies, NCER funding may be appropriate for continuous improvement methods that are applied within a longer-term research project.
  • Disseminate Information that is Useful to Policymakers: TWG members agreed that NCER research results should be relevant and presented in easy-to-read formats tailored for specific stakeholder audiences. To address this issue, IES can create easy-to-understand research syntheses, as education leaders do not have time or training to comb through the results of individual studies. In addition, TWG members identified a need for research on what is needed for practitioners to translate research to practice, to support decision making, and to address barriers to implementation.
  • Attend to Equity in Grantmaking and Research Focus: TWG members were concerned about equity in grantmaking and broadening participation in the research process. One way to address this is to provide more structured technical assistance to ensure applicants new to IES funding develop competitive research proposals. Active outreach can also help to encourage experts most likely to address equity research questions to apply. TWG members also pointed out that interdisciplinary research teams could help unearth embedded inequities in data collection, measures, and models.

In addition to the ideas discussed above, TWG members suggested specific ideas for how NCER could support research that leads to policy and systems improvement. These are too numerous to include here, but they are described in full in the TWG summary report.


For questions about this blog or the TWG summary, please contact Corinne.Alfeld@ed.gov, NCER Program Officer for the Improving Education Systems topic.

Information and Advising: Identifying Effective Strategies that Help Students Navigate Postsecondary Education

The college pipeline from start to finish is an extraordinarily complex process with numerous decision points, options, and obstacles. Students from advantaged social backgrounds are more likely than those from disadvantaged backgrounds to attend schools and colleges staffed with advisers and support staff that have time and resources to assist them. They may also draw on relationships with family, adults in their communities, or knowledgeable peers for assistance in navigation decisions. For students without such supports, the sequence of choices may become so overwhelming that they respond by delaying decisions or making poor choices that lead to sizable delays in their degree progression. With these challenges in mind, IES has funded three information and advising initiatives that draw on insights from researchers, practitioners, and the research literature.

Technical Working Group Meeting

In July 2019, NCER convened a technical working group of 14 researchers and practitioners for a set of conversations structured around three intervention strategies that have garnered substantial attention over the last 5 years: nudges and other light-touch informational campaigns; intensive, proactive coaching and advising; and comprehensive approaches that comprise advising and other supports such as technology and financial incentives. Researchers and practitioners shared perceptions about the effectiveness of each strategy, its relevance to targeted student populations, and conditions for implementation. At the end of the day, working group members provided recommendations (see the Technical Working Group Meeting Summary for a full list), including the following:  

  • Institutions should help determine what strategies get tested, apply for research grants, and participate in the research as it progresses.
  • Research is needed that addresses the large amount of information that students face and that identifies the types of information that students respond to and act on.
  • Replication studies should be designed to measure the effectiveness of promising intervention strategies for specific student groups, with the goal of enhancing effectiveness and cost-effectiveness.

Practice Guide on Effective Advising for Postsecondary Students

In October 2021, the What Works Clearinghouse (WWC) released a Practice Guide on Effective Advising for Postsecondary Students. The practice guide includes four evidence-based recommendations designed for an audience of administrators and staff at community colleges, 4-year institutions, and other public or private technical colleges who are responsible for designing and/or delivering advising to students:

  • Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals.
  • Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career.
  • Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students’ achievement and progression.
  • Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement.

Gap Analysis of Information and Advising Research and Practice

In March 2020, the Lead Team of the College Completion Network began a project aimed at identifying gaps in the research evidence base for information and advising strategies. The project is organized into three parts:

  1. A systematic review of the research literature, documenting evidence of the effect of information and advising policies, practices, and programs on student outcomes
  2. A scan of information and advising policies, practices, and programs that colleges use to improve student outcomes
  3. A gap analysis to compare the findings from the scan to the findings from the systematic review to look for effective practices that are not widely implemented and promising practices in the field that have not been evaluated

The team plans to report its full set of findings by December 2021. College Completion Network study descriptions are available here: https://collegecompletionnetwork.org/studies.


This blog is the second in a blog series on Effective Postsecondary Interventions that highlights interventions with evidence of effectiveness generated through IES-funded research. For the first blog in the series, please see here.

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division, and Felicia Sanders (Felicia.Sanders@ed.gov), a Program Officer for the What Works Clearinghouse within NCEE’s Knowledge Use Division.

 

Working to Understand the Policy Process in the Development of Michigan’s Read by Grade Three Law

In recent decades, there has been an emphasis on quantitative, causal research in education policy. These methods are best suited for answering questions about the effects of a policy and whether it achieved its intended outcomes. While the question of, “Did it work?” remains critical, there is a need for research that also asks, “Why did it work? For whom? In what contexts?” To answer these types of questions, researchers must incorporate rigorous qualitative methods into their quantitative studies. Education research organizations like the Association for Education Finance and Policy have explicitly invited proposals using qualitative and mixed methodologies in an effort to elevate research addressing a range of critical education policy questions. Funding organizations, including IES, encourage applicants to incorporate qualitative methods into their research process. In this guest blog, Amy Cummings, Craig De Voto, and Katharine Strunk discuss how they are using qualitative methods in their evaluation of a state education policy.

 

In our IES-funded study, we use qualitative, survey, and administrative data to understand the implementation and impact of Michigan’s early literacy law—the Read by Grade Three Law. Like policies passed in 19 other states, the Read by Grade Three Law aims to improve K-3 student literacy skills and mandates retention for those who do not meet a predetermined benchmark on the state’s third-grade English language arts assessment. Although the retention component of these policies remain controversial, similar laws are under consideration in several other states, including Alaska, Kentucky, and New Mexico. Below are some of the ways that we have integrated qualitative methods in our evaluation study to better understand the policy process in the development of the Read by Grade Three Law.  

Collecting qualitative sources helped us understand how the policy came to be, thereby assisting in the structure of our data collection for examining the law’s implementation and subsequent effects. In our first working paper stemming from this study, we interviewed 24 state-level stakeholders (policymakers, state department of education officials, early literacy leaders) involved in the development of the law and coded state policy documents related to early literacy to assess the similarity between Michigan’s policy and those of other states. Understanding the various components of the Law and how they ended up in the policy led us to ensure that we asked educators about their perceptions and implementation of these components in surveys that are also part of our evaluation. For example, because our interviews made clear the extent to which the inclusion of the retention component of the Law was controversial during its development, we included questions in the survey to assess educators’ perceptions and intended implementation of this component of the Law. In addition, it confirmed the importance of our plan to use state administrative retention and assessment data to evaluate the effect of retention on student literacy outcomes.

To trace the Read by Grade Three Law’s conception, development, and passage, we analyzed these qualitative data using two theories of the policy process: Multiple Streams Framework (MSF) and policy transfer. MSF says that policy issues emerge on government agendas through three streams: problem, policy, and political. When these streams join, a policy window is opened during which there is a greater opportunity for passing legislation. Meanwhile, policy transfer highlights how policies enacted in one place are often used in the development of policies in another.

We found that events in the problem and political streams created conditions ripe for the passage of an early literacy policy in Michigan:

  • A national sentiment around improving early literacy, including a retention-based third-grade literacy policy model that had been deemed successful in Florida
  • A pressing problem took shape, as evidenced by the state’s consistently below average fourth-grade reading scores on the National Assessment of Educational Progress
  • A court case addressing persistently low-test scores in a Detroit-area district
  • Previous attempts by the state to improve early literacy

As a result of these events, policy entrepreneurs—those willing to invest resources to get their preferred policy passed—took advantage of political conditions in the state and worked with policymakers to advance a retention-based third-grade literacy policy model. The figure below illustrates interviewee accounts of the Read by Grade Three Law’s development. Our policy document analysis further reveals that Michigan’s and Florida’s policies are very similar, only diverging on nine of the 50 elements on which we coded.

 

 

Although this study focuses on the development and passage of Michigan’s early literacy law, our findings highlight both practical and theoretical elements of the policy process that can be useful to researchers and policymakers. To this end, we show how particular conditions, coupled by policy entrepreneurs, spurred Michigan’s consideration of such a policy. It is conceivable that many state education policies beyond early literacy have taken shape under similar circumstances: a national sentiment combined with influential brokers outside government. In this way, our mixed-methods study provides a practical model of what elements might manifest to enact policy change more broadly.

From a theoretical standpoint, this research also extends our understanding of the policy process by showing that MSF and the theory of policy transfer can work together. We learned that policy entrepreneurs can play a vital role in transferring policy from one place to another by capitalizing on conditions in a target location and coming with a specific policy proposal at the ready.

There is, of course, more to be learned about the intersection between different theories of the policy process, as well as how external organizations as opposed to individuals operate as policy entrepreneurs. As the number of education advocacy organizations continues to grow and these groups become increasingly active in shaping policy, this will be an exciting avenue for researchers to continue to explore.

This study is just one example of how qualitative research can be used in education policy research and shows how engaging in such work can be both practically and theoretically valuable. The most comprehensive evaluations will use different methodologies in concert with one another to understand education policies, because ultimately, how policies are conceptualized and developed has important implications for their effectiveness.


Amy Cummings is an education policy PhD student and graduate research assistant at the Education Policy Innovation Collaborative (EPIC) at Michigan State University (MSU).

Craig De Voto is a visiting research assistant professor in the Learning Sciences Research Institute at the University of Illinois at Chicago and an EPIC affiliated researcher.

Katharine O. Strunk is the faculty director of EPIC, the Clifford E. Erickson Distinguished Chair in Education, and a professor of education policy and by courtesy economics at MSU.

Effective Postsecondary Interventions: Early Colleges Combine High School and College to Benefit Students

Across a set of research grant programs, IES generates knowledge of how to increase students’ access to, progress through, and completion of postsecondary credentials and degrees. Funded projects develop and test a range of interventions from state-level policies to classroom practices, with an emphasis on strategies that promote college attainment for students historically underrepresented in postsecondary education. This new blog series called Effective Postsecondary Interventions highlights interventions with evidence of effectiveness generated through IES-funded research.

 

The Early College High School Model

The Early College High School (ECHS) model addresses barriers to college attainment commonly experienced by students historically underrepresented in higher education. Students from low-income families and minoritized racial and ethnic groups often attend high schools that lack rigorous pre-college courses, strong support for college enrollment, and established connections to colleges and universities. For those students, cost is an additional barrier to enrollment and persistence in college. The ECHS model addresses these barriers by combining secondary and postsecondary instruction within the same school, prioritizing college-preparatory high school courses, offering opportunities to enroll in college courses while in high school, and providing comprehensive supports for academic and social-emotional development—all at little or no cost to the students. Although concurrent enrollment in high school and college courses (dual enrollment) is a core component of the model, the ECHS model is more expansive than dual enrollment because it includes a broader set of intervention components and has a clear equity objective.

All early colleges reflect these four design principles:

  • Enrollment of students historically underrepresented in higher education
  • Partnerships including a local education agency, a higher education institution, and the surrounding community
  • An integrated program of secondary and postsecondary education with the goal of all students earning 1 to 2 years of college credit prior to high school graduation
  • A comprehensive support system for students to develop academic skills as well as social and behavioral skills.

 

Attending Early Colleges Increases Postsecondary Attainment

Four IES-funded projects have evaluated impacts of the ECHS model. Prior studies within these projects found that significantly larger percentages of early college students completed a college preparatory course of study during high school and enrolled in postsecondary education within six years of entering high school. The two most recent projects assess postsecondary attainment:

In addition, early college students earned associate degrees at rates that exceeded their counterparts in traditional high schools by 22% and 18%, respectively, while earning bachelor’s degrees at equal or higher rates. These impacts are substantial, and especially noteworthy because both evaluations studied early colleges across a range of settings. Moreover, the impacts are similar for different student subgroups, regardless of gender, race/ethnicity, or family income.

 

Why have early colleges been so effective?

Early colleges set high expectations, provide high-quality interactions between staff and students, and encourage college access and success for all students. Several studies confirm that early colleges substantially improve high school experiences and outcomes. Students in North Carolina early colleges reported higher expectations, more rigorous and relevant instruction, stronger academic and social supports, and better relationships with teachers than their counterparts in other high schools. Early college students in AIR’s five-state sample reported significantly higher levels of college-going culture and instructor support than their counterparts in traditional high schools. The combination of high expectations and supportive relationships promotes better outcomes for early college students beginning in ninth grade (compared with students in other high schools). For instance, early college students are more likely to persist in college-preparatory math courses, attain a significant number of college credits during high school, and graduate from high school. Importantly, these positive results hold for students from all racial and ethnic groups, including students who enter high school at low levels of math proficiency.


For more information about the studies, the SERVE Center at the University of North Carolina at Greensboro has posted a brief of impact findings from their evaluation of North Carolina early colleges. The American Institutes of Research has posted a brief of impact findings from their five-state evaluation of early colleges.  

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division.