Inside IES Research

IES is pleased to announce the National Education Research and Development Center for Improving Rural Postsecondary Education. This Center will be the first rural R&D center to focus on improving access to postsecondary education and completion of postsecondary degrees and credentials for students from rural K-12 districts and locales. Center researchers will disaggregate findings about rural students to better understand variation by student subgroups including racial/ethnic groupings, levels of family income, and genders. Through this investment, IES will support research useful to leaders and staff in rural districts and high schools, administrators and practitioners at rural-serving colleges and universities, and state-level administrators and policymakers concerned with extending postsecondary opportunities to rural students.

This investment continues IES’s prior and ongoing investments in rural R&D centers which started in 2004 with the National Research Center on Rural Education Support (NCRES), and continued in 2009 with The National Center for Research on Rural Education. IES currently supports two rural R&D Centers: The National Center for Rural Education Research Networks (NCRERN), and The National Center for Rural School Mental Health (NCRSMH): Enhancing the Capacity of Rural Schools to Identify, Prevent, and Intervene in Youth Mental Health Concerns. Specifically, this new Center expands on the work of NCRES, which explored the factors that influence postsecondary aspirations for rural African American, Latinx, and Native American high school students. One of the new Center’s eight studies will conduct a representative survey of rural students in three states to assess their postsecondary aspirations and choices.

The new rural Center plans an expansive research agenda that includes a national landscape study of rural students’ postsecondary enrollment and factors that contribute to their postsecondary success, drawing on National Center for Education Statistics data including district-level data collected through the Common Core of Data program and nationally-representative student-level data collected through the High School Longitudinal Study of 2009. Six additional studies conducted across 10 states focus on rural students and the programs that support them in their transitions to and through college. In addition to the descriptive study of rural students aspirations mentioned above, five studies assess specific strategies for improving postsecondary access and success including dual enrollment programs for rural high school students, community-based organizations that encourage and facilitate college enrollment, supportive high school and postsecondary environments for African American students, the comprehensive Montana 10 student support program, and a train-in-place program for rural nursing students.

The Center will carry out a robust program of national leadership and dissemination. Leadership activities will include building the capacity of state agencies, rural-located practitioners, and early-career researchers to conduct research on rural students and rural-serving colleges and universities. Representatives from six state commissions and college systems, and advisory panels of external researchers and practitioners from across the country will guide the Center’s research. Five national organizations that partner with postsecondary institutions and advocate for student needs will assist the Center with disseminating its findings to broad audiences of policymakers, administrators, and practitioners that serve rural students and districts. At the conclusion of its work, the Center will publish a synthesis of its research findings and share it widely.

Map of Rural Postsecondary Education R&D Center Focus States

This blog was written by James Benson (James.Benson@ed.gov), program officer, NCER. 

State financial aid programs have the potential to substantially augment the support that students receive from the federal Pell Grant. Federal programs, most notably the Federal Pell Grant program, have historically played the lead role of providing a solid foundation of financial support to students, with states playing the supporting role of providing additional aid to students who meet specific eligibility requirements. In recent years, states have moved to innovate their financial aid programs in ways that have the potential to increase total aid packages, meet a wider range of needs, and serve a broader population of students. The effects of these recent innovations are mostly unknown yet of great interest to state legislators and policymakers. To address this issue, IES is funding a set of five research projects that assess the scope and effects of innovative financial aid programs in California, Connecticut, Michigan, Tennessee, and Washington state. This blog describes how the five projects are contributing to the evidence base.

State financial aid program eligibility rules differ in ways that can substantially alter total aid awards, the scope of the population that can be served, and the ways in which students can use aid funds to meet their various needs while enrolled in college. For example, one key policy attribute that affects the total aid award is whether awards are calculated independently of the Pell Grant­–as “first-dollar” awards that add to the Pell award if state eligibility requirements are met– or as “last-dollar” awards that supplement Pell awards conditional upon eligibility and appropriate-use requirements. Policies including an eligibility requirement for recent high school graduation within the state tend to limit aid access for older and returning students. In addition, financial need requirements can limit or broaden the pool of eligible recipients, depending on family income thresholds. Policies that require completion of the federal FAFSA Form without offering an alternative state application tend to close off access to aid for undocumented immigrants. Merit and high school GPA requirements can close off aid access to students who are otherwise ready for college. Moreover, appropriate-use requirements in some states limit aid usage to tuition and registration expenses while other states allow aid usage for living expenses such as housing and transportation.

Given these variations in program eligibility rules, state officials want to know if their aid programs are reaching targeted student groups, meeting their needs in ways that allow them to focus on their studies, and making a difference in their academic and subsequent labor market outcomes. In an effort to support decision making, IES is funding five projects that are each working closely with state officials to understand the features of their programs and conducting research to assess which students are accessing the programs, the extent of support provided by the programs, and their effects on enrollment in and progression through college. Below is the list of the IES-funded projects.

• Evaluating Incentives for Full-time Enrollment at California Community Colleges: Researchers from the Wheelhouse Center for Community College Leadership and Research at the University of California-Davis are assessing the effects of the Student Success Completion Grant, a supplement to California’s Cal Grant B financial aid program, on community college enrollment intensity and academic outcomes. This award provides additional financial aid for full-time enrollment to encourage timely degree completion. 
• Early First-Dollar Categorical Need-Based Aid: A New Model for Making College Affordable?: Researchers at the University of Michigan Gerald R. Ford School of Public Policy are assessing the effects of Michigan’s Tuition Incentive Program (TIP), the state’s largest financial aid program. TIP provides a first-dollar scholarship which fully covers in-district community college tuition and fees and provides partial tuition support at four-year institutions, for low-income students as indicated by Medicaid participation before high school graduation.
• Measuring the Impact of State Financial Aid Programs: Researchers at the Connecticut Office of Higher Education and the Connecticut Office of Policy and Management are leading a project to assess the enrollment and postsecondary attainment effects of the Roberta B. Willis Scholarship Program. This last-dollar program has need-merit and need-based components that supplement the Pell Grant to cover the costs of tuition and fees and provide a stipend for books and supplies, for low-income in-state recent high school graduates.
• A Collaborative Use of a State Longitudinal Data System: A Study of the Washington College Grant Program: Researchers from MDRC and the Washington Student Achievement Council are assessing the effects of the Washington College Grant. This first-dollar scholarship is available to eligible low- and middle-income students of all ages enrolled in registered apprenticeships as well as two- and four-year degree programs.
• Studying Heterogeneity in the Impact of Merit Aid Receipt and Loss on Academic and Labor Market Outcomes: Researchers at University of Michigan are partnering with the Tennessee Higher Education Commission, the Tennessee Student Assistance Commission, and the Tennessee Board of Regents to assess the Tennessee Hope Scholarship. This merit-based program awards financial aid to graduates of Tennessee high schools who enroll in a 2- or 4-year public or private postsecondary institution within 16 months of high school graduation, conditional on meeting high school achievement requirements.

We are excited to fund these projects and look forward to the findings they will be sharing, starting in fall 2024.

This blog was written by James Benson (James.Benson@ed.gov), program officer in the Policy and Systems team at NCER.

With rapid technological advances, the U.S. labor market exhibits a growing need for more frequent and ongoing skill development. Community college noncredit career and technical education (CTE) programs that allow students to complete workforce training and earn credentials play an essential role in providing workers with the skills they need to compete for jobs in high-demand fields. Yet, there is a dearth of research on these programs because noncredit students are typically not included in state and national postsecondary datasets. In this guest blog for CTE Month, researchers Di Xu, Benjamin Castleman, and Betsy Tessler discuss their IES-funded exploration study in which they build on a long-standing research partnership with the Virginia Community College System and leverage a variety of data sources to investigate the Commonwealth’s FastForward programs. These programs are noncredit CTE programs designed to lead to an industry-recognized credential in one of several high-demand fields identified by the Virginia Workforce Board.

In response to the increasing demand for skilled workers in the Commonwealth, the Virginia General Assembly passed House Bill 66 in 2016 to establish the New Economy Workforce Credential Grant Program (WCG) with the goal of providing a pay-for-performance model for funding noncredit training. The WCG specifically funds FastForward programs that lead to an industry-recognized credential in a high-demand field in the Commonwealth. Under this model, funding is shared between the state, students, and training institutions based on student performance, with the goal of ensuring workforce training is affordable for Virginia residents. An important implication of WCG is that it led to systematic, statewide collection of student-level data on FastForward program enrollment, program completion, industry credential attainment, and labor market performance. Drawing on these unique data, coupled with interviews with key stakeholders, we generated findings on the characteristics of FastForward programs, as well as the academic and labor market outcomes of students enrolled in these programs. We describe the preliminary descriptive findings below.

FastForward programs enroll a substantially different segment of the population from credit-bearing programs and offer a vital alternative route to skill development and workforce opportunities, especially for demographic groups often underrepresented in traditional higher education. FastForward programs in Virginia enroll a substantially higher share of Black students, male students, and older students than short-duration, credit-bearing programs at community colleges that typically require one year or less to complete. Focus groups conducted with FastForward students at six colleges indicate that the students were a mix of workers sent by their employers to learn specific new skills and students who signed up for a FastForward program on their own. Among the latter group were older career changers and recent high school graduates, many of whom had no prior college experience and were primarily interested in landing their first job in their chosen field. Moreover, 61% of FastForward participants have neither prior nor subsequent enrollment in credit-bearing programs, highlighting the program’s unique role in broadening access to postsecondary education and career pathways.

FastForward programs offer an alternative path for students who are unsuccessful in credit-bearing programs. The vast majority of students (78%) enrolled in only one FastForward program, with the average enrollment duration of 1.5 quarters, which is notably shorter than most traditional credit-bearing programs. While 36% have prior credit-bearing enrollment, fewer than 20% of these students earned a degree or certificate from it, and less than 12% of FastForward enrollees transitioned to credit-bearing training afterward. Interviews with administrators and staff indicated that while some colleges facilitate noncredit-to-credit pathways by granting credit for prior learning, others prioritize employment-focused training and support over stackable academic pathways due to students’ primary interest in seeking employment post-training.

FastForward programs have a remarkable completion rate and are related to high industry credential attainment rates. Over 90% of students complete their program, with two-thirds of students obtaining industry credentials. Student focus groups echoed this success. They praised the FastForward program and colleges for addressing both their tuition and non-tuition needs. Many students noted that they had not envisioned themselves as college students and credited program staff, financial aid, and institutional support with helping them to be successful.

Earning an industry credential through FastForward on average increases quarterly earnings by approximately $1,000. In addition, industry credentials also increase the probability of being employed by 2.4 percentage points on average. We find substantial heterogeneity in economic return across different fields of study, where the fields of transportation (for example, commercial driver’s license) and precision production (for example, gas metal arc welding) seem to be associated with particularly pronounced earnings premiums. Within programs, we do not observe significant heterogeneity in economic returns across student subgroups.

What’s Next?

In view of the strong economic returns associated with earning an industry credential and the noticeable variation in credential attainment between training institutions and programs, our future exploration intends to unpack the sources of variation in program-institution credential attainment rates and to identify specific program-level factors that are within the control of an institution and which are associated with higher credential rates and lower equity gaps. Specifically, we will collect additional survey data from the top 10 most highly-enrolled programs at the Virginia Community College System (VCCS) that will provide more nuanced program-level information and identify which malleable program factors are predictive of higher credential attainment rates, better labor market outcomes, and smaller equity gaps associated with these outcomes.

Di Xu is an associate professor in the School of Education at UC, Irvine, and the faculty director of UCI’s Postsecondary Education Research & Implementation Institute.

Ben Castleman is the Newton and Rita Meyers Associate Professor in the Economics of Education at the University of Virginia.

Betsy Tessler is a senior associate at MDRC in the Economic Mobility, Housing, and Communities policy area.

Note: A team of researchers, including Kelli Bird, Sabrina Solanki, and Michael Cooper contributed jointly to the quantitative analyses of this project. The MDRC team, including Hannah Power, Kelsey Brown, and Mark van Dok, contributed to qualitative data collection and analysis. The research team is grateful to the Virginia Community College System (VCCS) for providing access to their high-quality data. Special thanks are extended to Catherine Finnegan and her team for their valuable guidance and support throughout our partnership.

This project was funded under the Postsecondary and Adult Education research topic; questions about it should be directed to program officer James Benson (James.Benson@ed.gov).

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), NCER program officer for the CTE research topic.

Through the Transformative Research in the Education Sciences Grants program (ALN 84.305T), IES  invests in innovative research that has the potential to make dramatic advances towards solving seemingly intractable problems and challenges in the education field, as well as to accelerate the pace of conducting education research to facilitate major breakthroughs. In the most recent FY 2024 competition for this program, IES invited applications from partnerships between researchers, product developers, and education agencies to propose transformative solutions to major education problems that leverage advances in technology combined with research insights from the learning sciences.

IES is thrilled to announce that three grants have been awarded in the FY 2024 competition. Building on 20 years of IES research funding to lay the groundwork for advances, these three projects focus on exploring potentially transformative uses of generative artificial intelligence (AI) to deliver solutions that can scale in the education marketplace if they demonstrate positive impacts on education outcomes. The three grants are:

Active Learning at Scale (Active L@S): Transforming Teaching and Learning via Large-Scale Learning Science and Generative AI

Awardee: Arizona State University (ASU; PI: Danielle McNamara)

The project team aims to solve the challenge that postsecondary learners need access to course materials and high-quality just-in-time generative learning activities flexibly and on-the-go.  The solution will be a mobile technology that uses interactive, research-informed, and engaging learning activities created on the fly, customized to any course content with large language models (LLMs). The project team will leverage two digital learning platforms from the SEERNet network – Terracotta and ASU Learning@Scale – to conduct research and will include over 100,000 diverse students at ASU, with replication studies taking place at Indiana University (IU). IES funding has supported a large portion of the research used to identify the generative learning activities the team will integrate into the system—note-taking, self-explanation, summarization, and question answering (also known as retrieval practice). The ASU team includes in-house technology developers and researchers, and they are partnering with researchers at IU and developers at INFLO and Clevent AI Technology LLC. The ASU and IU teams will have the educator perspective represented on their teams, as these universities provide postsecondary education to large and diverse student populations.

Talking Math: Improving Math Performance and Engagement Through AI-Enabled Conversational Tutoring

Awardee: Worcester Polytechnic Institute (PI: Neil Heffernan)

The project team aims to provide a comprehensive strategy to address persistent achievement gaps in math by supporting students during their out-of-school time. The team will combine an evidence-based learning system with advances in generative AI to develop a conversational AI tutor (CAIT– pronounced as “Kate”) to support independent math practice for middle school students who struggle with math, and otherwise, may not have access to after-school tutoring. CAIT will be integrated into ASSISTments, a freely available, evidence-based online math platform with widely used homework assignments from open education resources (OER). This solution aims to dramatically improve engagement and math learning during independent math problem-solving time. The team will conduct research throughout the product development process to ensure that CAIT is effective in supporting math problem solving and is engaging and supportive for all students. ASSISTments has been used by over 1 million students and 30,000 teachers, and IES has supported its development and efficacy since 2003. The project team includes researchers and developers at Worcester Polytechnic Institute and the ASSISTments Foundation, researchers from WestEd, educator representation from Greater Commonwealth Virtual School, and a teacher design team.

Scenario-Based Assessment in the age of generative AI: Making space in the education market for alternative assessment paradigm

Awardee: University of Memphis (PI: John Sabatini)

Educators face many challenges building high-quality assessments aligned to course content, and traditional assessment practices often lack applicability to real world scenarios. To transform postsecondary education, there needs to be a shift in how knowledge and skills are assessed to better emphasize critical thinking, complex reasoning, and problem solving in practical contexts. Supported in large part by numerous IES-funded projects, including as part of the Reading for Understanding Initiative, the project team has developed a framework for scenario-based assessments (SBAs). SBAs place knowledge and skills into a practical context and provide students with the opportunity to apply their content knowledge and critical thinking skills. The project team will leverage generative AI along with their framework for SBAs to create a system for postsecondary educators to design and administer discipline-specific SBAs with personalized feedback to students, high levels of adaptivity, and rich diagnostic information with little additional instructor effort. The project team includes researchers, developers, and educators at University of Memphis and Georgia State University, researchers and developers at Educational Testing Service (ETS), and developers from multiple small businesses including Capti/Charmtech, MindTrust, Caimber/AMI, and Workbay who will participate as part of a technical advisory group.

We are excited by the transformative potential of these projects and look forward to seeing what these interdisciplinary teams can accomplish together. While we are hopeful the solutions they create will make a big impact on learners across the nation, we will also share lessons learned with the field about how to build interdisciplinary partnerships to conduct transformative research and development.

For questions or to learn more about the Transformative Research in the Education Sciences grant program, please contact Erin Higgins (Erin.Higgins@ed.gov), Program Lead for the Accelerate, Transform, Scale Initiative.

The Department of Education’s Small Business Innovation Research Program (SBIR), which IES administers, funds the research, development, and evaluation of new, commercially viable education technology products. Time Tails is an online game intended to prepare middle and high school students for success in postsecondary education and career pathways in computer science. The game, which introduces students to coding within the context of computer game design, was developed as part of the SBIR project Coding Bridge: Bridging Computer Science for Girls. In this interview blog, game developers Grace Collins and Carrie Linden of Liminal eSports (now called Snowbright Studio) discuss Time Tails and the importance of inspiring female-identifying students to code.   

What is Time Tails?

Carrie:

Time Tails is a series of digital games funded by the U.S. Department of Education and the Tides Foundation to help provide learners of all ages with an entry point into learning game design. Each episode transports you to a different point in history, where you help Ari and Zoe (two rad cartoon cats from the 1980s) fix glitches in history while also learning and practicing 3D game development and game design skills. The games are packed full of 80s’ puns, humor, references to salmon (it is a game about cats, after all), and story, while also encouraging players to learn about some amazing folx that sometimes get left out of history class textbooks.

Grace:

For me, Time Tails is a tool. It’s that missing bridge. Imagine you have a student who is interested in design who has been playing around in Scratch for years. When you show them Unity or Unreal game engines (popular game architectures), they may balk at the complexity of them. Time Tails breaks down those complex interfaces into digestible components that gives students the confidence and interest in making the jump across the gap. We’re continuing to release new Time Tails episodes every six months or so, adding new historical periods and new technical concepts. 

Thanks to our partnership with Unity, we’ve also been able to create an entire year’s curriculum for AP Computer Science Principles aligned to College Board’s standards and the International Society for Technology in Education (ISTE) standards.

What inspired you to create Time Tails?

Grace:

I was teaching computer science at an all-girls school in Cleveland, and the lack of resources for my students was just painfully obvious. There are a lot of generic coding resources, but when I went looking for something that could creatively inspire them and also lay the foundation for a career in game development or real-time 3D development, I was always coming up short.

Some of the first iterations of Time Tails were done right there in my classroom as I asked my students what they cared about (underrepresented female and LGBTQ+ figures in history), what they liked (cats), and even how different colors make them more or less likely to engage with learning content. When learning software pops up and it’s all steel gray and black, my students would look at it and say, “That looks like it’s for my brother.” They knew, and we know too, who the audience is for some of these tools out there. Time Tails tries to do all of that differently.

Carrie:

When you look at data on who is currently working in the computer science and game development industries, you will find that men are overwhelmingly the ones with active roles in the field. When we looked at where these drop off points were for girls in computer science career pathways, we found some interesting things. Girls were often leaving coding and computer science before they made it to high school, and most schools offered little in computer science and coding instruction during that gap between entry level software (like Scratch) and full game development platforms (like Unity). There was clearly a need for something to bridge that gap between tools used by younger kids and professional developers. There was also a need for those tools to be welcoming to female identifying and gender diverse youth as they are the ones falling out of the career.

What elements of Time Tails are uniquely tailored to female-identifying students? 

Carrie:

We decided to build Time Tails around narrative. As you progress, you get pulled into the written story and learn more about our feline heroes Ari and Zoe along with the people that they are helping. 

Humor, color, and charm also all tested well with our target audience compared to the typically dry YouTube tutorials and guides that you see out there that covers similar material. We packed our game full of ’80s puns and silly jokes, seasonal allergies (relatable), and made sure that each level was filled with colorful art and adorable characters. More often than not, these characters are strong women from history whose stories don't frequently make it into the textbooks. Our leading cats are female identifying and nonbinary, making Time Tails the first ever learning game featuring a canonically nonbinary character.

Grace:

It can be hard sometimes reading interpretations of our work. Adults will come in saying that the game looks too young or too childish. They want it to be more mature. They want darker colors and a more serious take. We can’t speak for everyone, but when we tested this game, that’s just not what teen girls wanted. They already had a lot of anxiety about getting into computer science in the first place. They’re VERY aware that it’s a masculine dominated field. They need that entry point that says it’s okay to be silly. They need to see others like themselves throughout history making waves. It’s been really heart-warming to see teens playing it, and even more so when their parent sits down with them to explain all of the 80s’ references. Those have been great moments as we’ve been out there testing this game.

What advice can you give other game developers who focus on female-identifying students?

Carrie:

Representation matters. If you can showcase the work that female-identifying folx are doing in the games industry, then you really should. Too often we see the tech and games industry primarily focusing on the women working (super important!) community management roles, but we also need to see highlights of the work done by female identifying developers, writers, quality assurance staff, producers, and more. Highlight all the roles and not just the ones that the industry has already decided are a “good fit” for women in tech and games. 

Grace:

My main advice to any developer is to involve your audience early and often. Have teens give you feedback on art, characters, concepts, everything. And don’t be afraid to see that those teens don’t agree. Be bold and brave in serving the students that you are trying to reach. Stay true to your vision and your audience.

What are the next steps for Time Tails and Snowbright Studio?

Carrie:

Time Tails is currently available on Steam. One purchase gets you access to all current episodes along with additional episodes releasing every six months or so. We are working with our partner, FableVision, to publish a version that allows for classroom licensing on their FableVision Games platform as well.

Grace:

Snowbright is also very active in the tabletop game industry, publishing cozy mystery role-playing games (RPGs) as well as card and board games. Our most recent Cozy Companion magazine actually took Ari and Zoe on a brand-new adventure to 1966 West Virginia as they learned about pollinators and cryptids in a mini-tabletop RPG.

Grace Collins (they/them) is the Founder/CEO of Snowbright Studio, a Cleveland-based LGBTBE certified game studio dedicated to publishing heartwarming games and experiences. Grace previously led games and education policy at the US Department of Education and later coordinated federal game policy across the executive branch. Prior to serving at the Department, they managed and developed educational game projects at the Smithsonian Institution. Grace has taught computer science and game design at multiple levels and was profiled by the Associated Press for founding the first esports team in the nation at an all-girls’ high school.

Carrie Linden (she/they) is the Communications Manager at Snowbright Studio, handling social media, websites, and the creation of official copy for the organization. Carrie has a Master’s in Education and seven years of experience teaching in LGBTQ+ friendly public-school programs and has her Certificate of Esports Management from UC Irvine. Carrie is an active member in the gaming and content creation community.

This blog is part of a 3-part Inside IES Research blog series on sexual orientation and gender identity in education research in observance of Pride month. The other posts discuss the feedback from the IES LGBTQI+ Listening and Learning session and encourage researchers to submit FY 2024 applications focused on the educational experiences and outcomes of LGBTQI+ identifying students.

This blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), NCER program officer and co-chair of the IES Diversity Council.