Inside IES Research

Notes from NCER & NCSER

IES is Investing in Research on Innovative Financial Aid Programs in Five States

State financial aid programs have the potential to substantially augment the support that students receive from the federal Pell Grant. Federal programs, most notably the Federal Pell Grant program, have historically played the lead role of providing a solid foundation of financial support to students, with states playing the supporting role of providing additional aid to students who meet specific eligibility requirements. In recent years, states have moved to innovate their financial aid programs in ways that have the potential to increase total aid packages, meet a wider range of needs, and serve a broader population of students. The effects of these recent innovations are mostly unknown yet of great interest to state legislators and policymakers. To address this issue, IES is funding a set of five research projects that assess the scope and effects of innovative financial aid programs in California, Connecticut, Michigan, Tennessee, and Washington state. This blog describes how the five projects are contributing to the evidence base.

State financial aid program eligibility rules differ in ways that can substantially alter total aid awards, the scope of the population that can be served, and the ways in which students can use aid funds to meet their various needs while enrolled in college. For example, one key policy attribute that affects the total aid award is whether awards are calculated independently of the Pell Grant­–as “first-dollar” awards that add to the Pell award if state eligibility requirements are met– or as “last-dollar” awards that supplement Pell awards conditional upon eligibility and appropriate-use requirements. Policies including an eligibility requirement for recent high school graduation within the state tend to limit aid access for older and returning students. In addition, financial need requirements can limit or broaden the pool of eligible recipients, depending on family income thresholds. Policies that require completion of the federal FAFSA Form without offering an alternative state application tend to close off access to aid for undocumented immigrants. Merit and high school GPA requirements can close off aid access to students who are otherwise ready for college. Moreover, appropriate-use requirements in some states limit aid usage to tuition and registration expenses while other states allow aid usage for living expenses such as housing and transportation.

Given these variations in program eligibility rules, state officials want to know if their aid programs are reaching targeted student groups, meeting their needs in ways that allow them to focus on their studies, and making a difference in their academic and subsequent labor market outcomes. In an effort to support decision making, IES is funding five projects that are each working closely with state officials to understand the features of their programs and conducting research to assess which students are accessing the programs, the extent of support provided by the programs, and their effects on enrollment in and progression through college. Below is the list of the IES-funded projects.

We are excited to fund these projects and look forward to the findings they will be sharing, starting in fall 2024.


This blog was written by James Benson (James.Benson@ed.gov), program officer in the Policy and Systems team at NCER.

Unlocking Opportunities: Understanding Connections Between Noncredit CTE Programs and Workforce Development in Virginia

With rapid technological advances, the U.S. labor market exhibits a growing need for more frequent and ongoing skill development. Community college noncredit career and technical education (CTE) programs that allow students to complete workforce training and earn credentials play an essential role in providing workers with the skills they need to compete for jobs in high-demand fields. Yet, there is a dearth of research on these programs because noncredit students are typically not included in state and national postsecondary datasets. In this guest blog for CTE Month, researchers Di Xu, Benjamin Castleman, and Betsy Tessler discuss their IES-funded exploration study in which they build on a long-standing research partnership with the Virginia Community College System and leverage a variety of data sources to investigate the Commonwealth’s FastForward programs. These programs are noncredit CTE programs designed to lead to an industry-recognized credential in one of several high-demand fields identified by the Virginia Workforce Board.

In response to the increasing demand for skilled workers in the Commonwealth, the Virginia General Assembly passed House Bill 66 in 2016 to establish the New Economy Workforce Credential Grant Program (WCG) with the goal of providing a pay-for-performance model for funding noncredit training. The WCG specifically funds FastForward programs that lead to an industry-recognized credential in a high-demand field in the Commonwealth. Under this model, funding is shared between the state, students, and training institutions based on student performance, with the goal of ensuring workforce training is affordable for Virginia residents. An important implication of WCG is that it led to systematic, statewide collection of student-level data on FastForward program enrollment, program completion, industry credential attainment, and labor market performance. Drawing on these unique data, coupled with interviews with key stakeholders, we generated findings on the characteristics of FastForward programs, as well as the academic and labor market outcomes of students enrolled in these programs. We describe the preliminary descriptive findings below.

FastForward programs enroll a substantially different segment of the population from credit-bearing programs and offer a vital alternative route to skill development and workforce opportunities, especially for demographic groups often underrepresented in traditional higher education. FastForward programs in Virginia enroll a substantially higher share of Black students, male students, and older students than short-duration, credit-bearing programs at community colleges that typically require one year or less to complete. Focus groups conducted with FastForward students at six colleges indicate that the students were a mix of workers sent by their employers to learn specific new skills and students who signed up for a FastForward program on their own. Among the latter group were older career changers and recent high school graduates, many of whom had no prior college experience and were primarily interested in landing their first job in their chosen field. Moreover, 61% of FastForward participants have neither prior nor subsequent enrollment in credit-bearing programs, highlighting the program’s unique role in broadening access to postsecondary education and career pathways.

FastForward programs offer an alternative path for students who are unsuccessful in credit-bearing programs. The vast majority of students (78%) enrolled in only one FastForward program, with the average enrollment duration of 1.5 quarters, which is notably shorter than most traditional credit-bearing programs. While 36% have prior credit-bearing enrollment, fewer than 20% of these students earned a degree or certificate from it, and less than 12% of FastForward enrollees transitioned to credit-bearing training afterward. Interviews with administrators and staff indicated that while some colleges facilitate noncredit-to-credit pathways by granting credit for prior learning, others prioritize employment-focused training and support over stackable academic pathways due to students’ primary interest in seeking employment post-training.

FastForward programs have a remarkable completion rate and are related to high industry credential attainment rates. Over 90% of students complete their program, with two-thirds of students obtaining industry credentials. Student focus groups echoed this success. They praised the FastForward program and colleges for addressing both their tuition and non-tuition needs. Many students noted that they had not envisioned themselves as college students and credited program staff, financial aid, and institutional support with helping them to be successful.

Earning an industry credential through FastForward on average increases quarterly earnings by approximately $1,000. In addition, industry credentials also increase the probability of being employed by 2.4 percentage points on average. We find substantial heterogeneity in economic return across different fields of study, where the fields of transportation (for example, commercial driver’s license) and precision production (for example, gas metal arc welding) seem to be associated with particularly pronounced earnings premiums. Within programs, we do not observe significant heterogeneity in economic returns across student subgroups.

What’s Next?

In view of the strong economic returns associated with earning an industry credential and the noticeable variation in credential attainment between training institutions and programs, our future exploration intends to unpack the sources of variation in program-institution credential attainment rates and to identify specific program-level factors that are within the control of an institution and which are associated with higher credential rates and lower equity gaps. Specifically, we will collect additional survey data from the top 10 most highly-enrolled programs at the Virginia Community College System (VCCS) that will provide more nuanced program-level information and identify which malleable program factors are predictive of higher credential attainment rates, better labor market outcomes, and smaller equity gaps associated with these outcomes.


Di Xu is an associate professor in the School of Education at UC, Irvine, and the faculty director of UCI’s Postsecondary Education Research & Implementation Institute.

Ben Castleman is the Newton and Rita Meyers Associate Professor in the Economics of Education at the University of Virginia.

Betsy Tessler is a senior associate at MDRC in the Economic Mobility, Housing, and Communities policy area.

Note: A team of researchers, including Kelli Bird, Sabrina Solanki, and Michael Cooper contributed jointly to the quantitative analyses of this project. The MDRC team, including Hannah Power, Kelsey Brown, and Mark van Dok, contributed to qualitative data collection and analysis. The research team is grateful to the Virginia Community College System (VCCS) for providing access to their high-quality data. Special thanks are extended to Catherine Finnegan and her team for their valuable guidance and support throughout our partnership.

This project was funded under the Postsecondary and Adult Education research topic; questions about it should be directed to program officer James Benson (James.Benson@ed.gov).

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), NCER program officer for the CTE research topic.

IES Makes Three New Awards to Accelerate Breakthroughs in the Education Field

Through the Transformative Research in the Education Sciences Grants program (ALN 84.305T), IES  invests in innovative research that has the potential to make dramatic advances towards solving seemingly intractable problems and challenges in the education field, as well as to accelerate the pace of conducting education research to facilitate major breakthroughs. In the most recent FY 2024 competition for this program, IES invited applications from partnerships between researchers, product developers, and education agencies to propose transformative solutions to major education problems that leverage advances in technology combined with research insights from the learning sciences.

IES is thrilled to announce that three grants have been awarded in the FY 2024 competition. Building on 20 years of IES research funding to lay the groundwork for advances, these three projects focus on exploring potentially transformative uses of generative artificial intelligence (AI) to deliver solutions that can scale in the education marketplace if they demonstrate positive impacts on education outcomes. The three grants are:

Active Learning at Scale (Active L@S): Transforming Teaching and Learning via Large-Scale Learning Science and Generative AI

Awardee: Arizona State University (ASU; PI: Danielle McNamara)

The project team aims to solve the challenge that postsecondary learners need access to course materials and high-quality just-in-time generative learning activities flexibly and on-the-go.  The solution will be a mobile technology that uses interactive, research-informed, and engaging learning activities created on the fly, customized to any course content with large language models (LLMs). The project team will leverage two digital learning platforms from the SEERNet networkTerracotta and ASU Learning@Scale – to conduct research and will include over 100,000 diverse students at ASU, with replication studies taking place at Indiana University (IU). IES funding has supported a large portion of the research used to identify the generative learning activities the team will integrate into the system—note-taking, self-explanation, summarization, and question answering (also known as retrieval practice). The ASU team includes in-house technology developers and researchers, and they are partnering with researchers at IU and developers at INFLO and Clevent AI Technology LLC. The ASU and IU teams will have the educator perspective represented on their teams, as these universities provide postsecondary education to large and diverse student populations.

Talking Math: Improving Math Performance and Engagement Through AI-Enabled Conversational Tutoring

Awardee: Worcester Polytechnic Institute (PI: Neil Heffernan)

The project team aims to provide a comprehensive strategy to address persistent achievement gaps in math by supporting students during their out-of-school time. The team will combine an evidence-based learning system with advances in generative AI to develop a conversational AI tutor (CAIT– pronounced as “Kate”) to support independent math practice for middle school students who struggle with math, and otherwise, may not have access to after-school tutoring. CAIT will be integrated into ASSISTments, a freely available, evidence-based online math platform with widely used homework assignments from open education resources (OER). This solution aims to dramatically improve engagement and math learning during independent math problem-solving time. The team will conduct research throughout the product development process to ensure that CAIT is effective in supporting math problem solving and is engaging and supportive for all students. ASSISTments has been used by over 1 million students and 30,000 teachers, and IES has supported its development and efficacy since 2003. The project team includes researchers and developers at Worcester Polytechnic Institute and the ASSISTments Foundation, researchers from WestEd, educator representation from Greater Commonwealth Virtual School, and a teacher design team.

Scenario-Based Assessment in the age of generative AI: Making space in the education market for alternative assessment paradigm

Awardee: University of Memphis (PI: John Sabatini)

Educators face many challenges building high-quality assessments aligned to course content, and traditional assessment practices often lack applicability to real world scenarios. To transform postsecondary education, there needs to be a shift in how knowledge and skills are assessed to better emphasize critical thinking, complex reasoning, and problem solving in practical contexts. Supported in large part by numerous IES-funded projects, including as part of the Reading for Understanding Initiative, the project team has developed a framework for scenario-based assessments (SBAs). SBAs place knowledge and skills into a practical context and provide students with the opportunity to apply their content knowledge and critical thinking skills. The project team will leverage generative AI along with their framework for SBAs to create a system for postsecondary educators to design and administer discipline-specific SBAs with personalized feedback to students, high levels of adaptivity, and rich diagnostic information with little additional instructor effort. The project team includes researchers, developers, and educators at University of Memphis and Georgia State University, researchers and developers at Educational Testing Service (ETS), and developers from multiple small businesses including Capti/Charmtech, MindTrust, Caimber/AMI, and Workbay who will participate as part of a technical advisory group.

We are excited by the transformative potential of these projects and look forward to seeing what these interdisciplinary teams can accomplish together. While we are hopeful the solutions they create will make a big impact on learners across the nation, we will also share lessons learned with the field about how to build interdisciplinary partnerships to conduct transformative research and development.


For questions or to learn more about the Transformative Research in the Education Sciences grant program, please contact Erin Higgins (Erin.Higgins@ed.gov), Program Lead for the Accelerate, Transform, Scale Initiative.

Inspiring and Teaching Girls to Code with Time Tails

The Department of Education’s Small Business Innovation Research Program (SBIR), which IES administers, funds the research, development, and evaluation of new, commercially viable education technology products. Time Tails is an online game intended to prepare middle and high school students for success in postsecondary education and career pathways in computer science. The game, which introduces students to coding within the context of computer game design, was developed as part of the SBIR project Coding Bridge: Bridging Computer Science for Girls. In this interview blog, game developers Grace Collins and Carrie Linden of Liminal eSports (now called Snowbright Studio) discuss Time Tails and the importance of inspiring female-identifying students to code.   

 

 

What is Time Tails?

Carrie:

Time Tails is a series of digital games funded by the U.S. Department of Education and the Tides Foundation to help provide learners of all ages with an entry point into learning game design. Each episode transports you to a different point in history, where you help Ari and Zoe (two rad cartoon cats from the 1980s) fix glitches in history while also learning and practicing 3D game development and game design skills. The games are packed full of 80s’ puns, humor, references to salmon (it is a game about cats, after all), and story, while also encouraging players to learn about some amazing folx that sometimes get left out of history class textbooks.

Grace:

For me, Time Tails is a tool. It’s that missing bridge. Imagine you have a student who is interested in design who has been playing around in Scratch for years. When you show them Unity or Unreal game engines (popular game architectures), they may balk at the complexity of them. Time Tails breaks down those complex interfaces into digestible components that gives students the confidence and interest in making the jump across the gap. We’re continuing to release new Time Tails episodes every six months or so, adding new historical periods and new technical concepts. 

Thanks to our partnership with Unity, we’ve also been able to create an entire year’s curriculum for AP Computer Science Principles aligned to College Board’s standards and the International Society for Technology in Education (ISTE) standards.

What inspired you to create Time Tails?

Grace:

I was teaching computer science at an all-girls school in Cleveland, and the lack of resources for my students was just painfully obvious. There are a lot of generic coding resources, but when I went looking for something that could creatively inspire them and also lay the foundation for a career in game development or real-time 3D development, I was always coming up short.

Some of the first iterations of Time Tails were done right there in my classroom as I asked my students what they cared about (underrepresented female and LGBTQ+ figures in history), what they liked (cats), and even how different colors make them more or less likely to engage with learning content. When learning software pops up and it’s all steel gray and black, my students would look at it and say, “That looks like it’s for my brother.” They knew, and we know too, who the audience is for some of these tools out there. Time Tails tries to do all of that differently.

Carrie:

When you look at data on who is currently working in the computer science and game development industries, you will find that men are overwhelmingly the ones with active roles in the field. When we looked at where these drop off points were for girls in computer science career pathways, we found some interesting things. Girls were often leaving coding and computer science before they made it to high school, and most schools offered little in computer science and coding instruction during that gap between entry level software (like Scratch) and full game development platforms (like Unity). There was clearly a need for something to bridge that gap between tools used by younger kids and professional developers. There was also a need for those tools to be welcoming to female identifying and gender diverse youth as they are the ones falling out of the career.

What elements of Time Tails are uniquely tailored to female-identifying students? 

Carrie:

We decided to build Time Tails around narrative. As you progress, you get pulled into the written story and learn more about our feline heroes Ari and Zoe along with the people that they are helping. 

Humor, color, and charm also all tested well with our target audience compared to the typically dry YouTube tutorials and guides that you see out there that covers similar material. We packed our game full of ’80s puns and silly jokes, seasonal allergies (relatable), and made sure that each level was filled with colorful art and adorable characters. More often than not, these characters are strong women from history whose stories don't frequently make it into the textbooks. Our leading cats are female identifying and nonbinary, making Time Tails the first ever learning game featuring a canonically nonbinary character.

Grace:

It can be hard sometimes reading interpretations of our work. Adults will come in saying that the game looks too young or too childish. They want it to be more mature. They want darker colors and a more serious take. We can’t speak for everyone, but when we tested this game, that’s just not what teen girls wanted. They already had a lot of anxiety about getting into computer science in the first place. They’re VERY aware that it’s a masculine dominated field. They need that entry point that says it’s okay to be silly. They need to see others like themselves throughout history making waves. It’s been really heart-warming to see teens playing it, and even more so when their parent sits down with them to explain all of the 80s’ references. Those have been great moments as we’ve been out there testing this game.

What advice can you give other game developers who focus on female-identifying students?

Carrie:

Representation matters. If you can showcase the work that female-identifying folx are doing in the games industry, then you really should. Too often we see the tech and games industry primarily focusing on the women working (super important!) community management roles, but we also need to see highlights of the work done by female identifying developers, writers, quality assurance staff, producers, and more. Highlight all the roles and not just the ones that the industry has already decided are a “good fit” for women in tech and games. 

Grace:

My main advice to any developer is to involve your audience early and often. Have teens give you feedback on art, characters, concepts, everything. And don’t be afraid to see that those teens don’t agree. Be bold and brave in serving the students that you are trying to reach. Stay true to your vision and your audience.

What are the next steps for Time Tails and Snowbright Studio?

Carrie:

Time Tails is currently available on Steam. One purchase gets you access to all current episodes along with additional episodes releasing every six months or so. We are working with our partner, FableVision, to publish a version that allows for classroom licensing on their FableVision Games platform as well.

Grace:

Snowbright is also very active in the tabletop game industry, publishing cozy mystery role-playing games (RPGs) as well as card and board games. Our most recent Cozy Companion magazine actually took Ari and Zoe on a brand-new adventure to 1966 West Virginia as they learned about pollinators and cryptids in a mini-tabletop RPG.


Grace Collins (they/them) is the Founder/CEO of Snowbright Studio, a Cleveland-based LGBTBE certified game studio dedicated to publishing heartwarming games and experiences. Grace previously led games and education policy at the US Department of Education and later coordinated federal game policy across the executive branch. Prior to serving at the Department, they managed and developed educational game projects at the Smithsonian Institution. Grace has taught computer science and game design at multiple levels and was profiled by the Associated Press for founding the first esports team in the nation at an all-girls’ high school.

Carrie Linden (she/they) is the Communications Manager at Snowbright Studio, handling social media, websites, and the creation of official copy for the organization. Carrie has a Master’s in Education and seven years of experience teaching in LGBTQ+ friendly public-school programs and has her Certificate of Esports Management from UC Irvine. Carrie is an active member in the gaming and content creation community.

This blog is part of a 3-part Inside IES Research blog series on sexual orientation and gender identity in education research in observance of Pride month. The other posts discuss the feedback from the IES LGBTQI+ Listening and Learning session and encourage researchers to submit FY 2024 applications focused on the educational experiences and outcomes of LGBTQI+ identifying students.

This blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), NCER program officer and co-chair of the IES Diversity Council.

Supporting Strategic Writers: The Use of Strategy Instruction and Genre Pedagogy in the Basic Writing Classroom

NCER student volunteer, Rachael Higham, has long been interested in writing instruction. She currently works as a remedial language tutor for high school students with disabilities, and she began her graduate studies with a focus on postsecondary writing instruction. To learn more about the current science and research on writing, Rachael interviewed Dr. Charles MacArthur about his research-based postsecondary writing curriculum, Supporting Strategic Writers (SSW), which he and his team developed and evaluated through IES grants. The goal of SSW is to foster metacognitive self-evaluation through the use of strategic learning and genre-based pedagogy to help improve writing skills and self-confidence.


Take a minute to answer this question: Do you remember how you were taught to write a paper in high school or in college?

Maybe you remember the five-paragraph essay, MLA formatting, or the RACE strategy, but were you ever taught specific strategies for planning and evaluating your papers?

While I was interviewing Dr. MacArthur about his recently completed IES project, he posed a similar question to me. He asked me how I navigated writing in college and if a teacher had ever explicitly taught me how to write. I realized that while I had some explicit teaching in text structure in high school, by the time I reached college, I relied heavily on feedback to inform my future writing. The idea that students learn from revising is a common view in writing education. However, this view does not always consider students who struggle with writing and who may need more explicit instruction, even in college.

As a teacher of high school students with learning disabilities, I often find that by the time many of my students reach my classroom, they feel defeated by the writing process. Writing is something that has become a source of fear and dread for them. My goal with each student is to find and develop strategies that bolster their writing skills and change writing from something that seems unattainable to something that they can do independently. I was excited to talk to Dr. MacArthur and learn more about the research that he and his have been doing. Below are his responses to the questions I posed.

What are the key components of the SSW curriculum?

The emphasis of SSW is to enable students to take control of their own learning through rhetorical analysis of genre. To do that, students are taught explicit strategies and cognitive procedures based on what good writers do. This is reinforced with metacognitive strategies that help students become aware of why they are using specific writing strategies and procedures and recognize how and when to transfer them to other classes. SSW places emphasis on genre-based strategies not only in the text but also in the planning and evaluation phases.

The heart of strategy instruction in SSW is the “think-aloud,” which is when instructors share, in real time, the thoughts that they are experiencing as they’re writing or editing a text to show how they are figuring things out. Instructors need to show—not just explain—how to write. What we writing instructors are teaching is invisible, so the think aloud makes the process visible to students. It also lets students see that writing is hard even for their teacher. Teachers can get stuck and need to work through it based on the strategies that are being taught.

What is the number one thing that you would tell a developmental or first-year writing teacher?

Teaching strategies to students on planning and evaluating their work helps improve writing. There have been hundreds of studies from K-12 (see these meta-analyses as examples 1, 2, 3) that show how strategy instruction works to improve writing. This experimental study of SSW adds to that literature and shows that strategic instruction with genre pedagogy can work in the postsecondary developmental writing environment.

What type of future research would you like to see done with the SSW curriculum?

There is a wealth of valuable research that could be done in the future. Future research could delve into how to build on the developmental course’s gain in subsequent courses. For example, it would be interesting to look at the transition between developmental writing courses and first-year composition in terms of pedagogical integration.

Another area of transfer is between compositions courses and disciplinary writing in postsecondary settings. For example, how could postsecondary institutions improve writing across the curriculum? How could strategy instruction similar to SSW work in this setting?

Additionally, strategy instruction started in special education, but it was found to be useful throughout the entire K-12 population. Similarly, SSW was found to be successful in developmental writing classrooms. It would be great to see the effects of SSW in first year composition classes.

You can find publications from this project and the earlier SSW project in ERIC here and here respectively. The What Works Clearinghouse also reviewed an earlier evaluation of the SSW here.


This blog was written by Rachael Higham, a graduate intern through the Virtual Student Federal Service Internships program, and facilitated by Dr. Meredith Larson (Meredith.Larson@ed.gov), a research analyst and program officer at NCER.