Inside IES Research

The Department of Education’s Small Business Innovation Research Program (SBIR), which IES administers, funds the research, development, and evaluation of new, commercially viable education technology products. Time Tails is an online game intended to prepare middle and high school students for success in postsecondary education and career pathways in computer science. The game, which introduces students to coding within the context of computer game design, was developed as part of the SBIR project Coding Bridge: Bridging Computer Science for Girls. In this interview blog, game developers Grace Collins and Carrie Linden of Liminal eSports (now called Snowbright Studio) discuss Time Tails and the importance of inspiring female-identifying students to code.   

What is Time Tails?

Carrie:

Time Tails is a series of digital games funded by the U.S. Department of Education and the Tides Foundation to help provide learners of all ages with an entry point into learning game design. Each episode transports you to a different point in history, where you help Ari and Zoe (two rad cartoon cats from the 1980s) fix glitches in history while also learning and practicing 3D game development and game design skills. The games are packed full of 80s’ puns, humor, references to salmon (it is a game about cats, after all), and story, while also encouraging players to learn about some amazing folx that sometimes get left out of history class textbooks.

Grace:

For me, Time Tails is a tool. It’s that missing bridge. Imagine you have a student who is interested in design who has been playing around in Scratch for years. When you show them Unity or Unreal game engines (popular game architectures), they may balk at the complexity of them. Time Tails breaks down those complex interfaces into digestible components that gives students the confidence and interest in making the jump across the gap. We’re continuing to release new Time Tails episodes every six months or so, adding new historical periods and new technical concepts. 

Thanks to our partnership with Unity, we’ve also been able to create an entire year’s curriculum for AP Computer Science Principles aligned to College Board’s standards and the International Society for Technology in Education (ISTE) standards.

What inspired you to create Time Tails?

Grace:

I was teaching computer science at an all-girls school in Cleveland, and the lack of resources for my students was just painfully obvious. There are a lot of generic coding resources, but when I went looking for something that could creatively inspire them and also lay the foundation for a career in game development or real-time 3D development, I was always coming up short.

Some of the first iterations of Time Tails were done right there in my classroom as I asked my students what they cared about (underrepresented female and LGBTQ+ figures in history), what they liked (cats), and even how different colors make them more or less likely to engage with learning content. When learning software pops up and it’s all steel gray and black, my students would look at it and say, “That looks like it’s for my brother.” They knew, and we know too, who the audience is for some of these tools out there. Time Tails tries to do all of that differently.

Carrie:

When you look at data on who is currently working in the computer science and game development industries, you will find that men are overwhelmingly the ones with active roles in the field. When we looked at where these drop off points were for girls in computer science career pathways, we found some interesting things. Girls were often leaving coding and computer science before they made it to high school, and most schools offered little in computer science and coding instruction during that gap between entry level software (like Scratch) and full game development platforms (like Unity). There was clearly a need for something to bridge that gap between tools used by younger kids and professional developers. There was also a need for those tools to be welcoming to female identifying and gender diverse youth as they are the ones falling out of the career.

What elements of Time Tails are uniquely tailored to female-identifying students? 

Carrie:

We decided to build Time Tails around narrative. As you progress, you get pulled into the written story and learn more about our feline heroes Ari and Zoe along with the people that they are helping. 

Humor, color, and charm also all tested well with our target audience compared to the typically dry YouTube tutorials and guides that you see out there that covers similar material. We packed our game full of ’80s puns and silly jokes, seasonal allergies (relatable), and made sure that each level was filled with colorful art and adorable characters. More often than not, these characters are strong women from history whose stories don't frequently make it into the textbooks. Our leading cats are female identifying and nonbinary, making Time Tails the first ever learning game featuring a canonically nonbinary character.

Grace:

It can be hard sometimes reading interpretations of our work. Adults will come in saying that the game looks too young or too childish. They want it to be more mature. They want darker colors and a more serious take. We can’t speak for everyone, but when we tested this game, that’s just not what teen girls wanted. They already had a lot of anxiety about getting into computer science in the first place. They’re VERY aware that it’s a masculine dominated field. They need that entry point that says it’s okay to be silly. They need to see others like themselves throughout history making waves. It’s been really heart-warming to see teens playing it, and even more so when their parent sits down with them to explain all of the 80s’ references. Those have been great moments as we’ve been out there testing this game.

What advice can you give other game developers who focus on female-identifying students?

Carrie:

Representation matters. If you can showcase the work that female-identifying folx are doing in the games industry, then you really should. Too often we see the tech and games industry primarily focusing on the women working (super important!) community management roles, but we also need to see highlights of the work done by female identifying developers, writers, quality assurance staff, producers, and more. Highlight all the roles and not just the ones that the industry has already decided are a “good fit” for women in tech and games. 

Grace:

My main advice to any developer is to involve your audience early and often. Have teens give you feedback on art, characters, concepts, everything. And don’t be afraid to see that those teens don’t agree. Be bold and brave in serving the students that you are trying to reach. Stay true to your vision and your audience.

What are the next steps for Time Tails and Snowbright Studio?

Carrie:

Time Tails is currently available on Steam. One purchase gets you access to all current episodes along with additional episodes releasing every six months or so. We are working with our partner, FableVision, to publish a version that allows for classroom licensing on their FableVision Games platform as well.

Grace:

Snowbright is also very active in the tabletop game industry, publishing cozy mystery role-playing games (RPGs) as well as card and board games. Our most recent Cozy Companion magazine actually took Ari and Zoe on a brand-new adventure to 1966 West Virginia as they learned about pollinators and cryptids in a mini-tabletop RPG.

Grace Collins (they/them) is the Founder/CEO of Snowbright Studio, a Cleveland-based LGBTBE certified game studio dedicated to publishing heartwarming games and experiences. Grace previously led games and education policy at the US Department of Education and later coordinated federal game policy across the executive branch. Prior to serving at the Department, they managed and developed educational game projects at the Smithsonian Institution. Grace has taught computer science and game design at multiple levels and was profiled by the Associated Press for founding the first esports team in the nation at an all-girls’ high school.

Carrie Linden (she/they) is the Communications Manager at Snowbright Studio, handling social media, websites, and the creation of official copy for the organization. Carrie has a Master’s in Education and seven years of experience teaching in LGBTQ+ friendly public-school programs and has her Certificate of Esports Management from UC Irvine. Carrie is an active member in the gaming and content creation community.

This blog is part of a 3-part Inside IES Research blog series on sexual orientation and gender identity in education research in observance of Pride month. The other posts discuss the feedback from the IES LGBTQI+ Listening and Learning session and encourage researchers to submit FY 2024 applications focused on the educational experiences and outcomes of LGBTQI+ identifying students.

This blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), NCER program officer and co-chair of the IES Diversity Council.

NCER student volunteer, Rachael Higham, has long been interested in writing instruction. She currently works as a remedial language tutor for high school students with disabilities, and she began her graduate studies with a focus on postsecondary writing instruction. To learn more about the current science and research on writing, Rachael interviewed Dr. Charles MacArthur about his research-based postsecondary writing curriculum, Supporting Strategic Writers (SSW), which he and his team developed and evaluated through IES grants. The goal of SSW is to foster metacognitive self-evaluation through the use of strategic learning and genre-based pedagogy to help improve writing skills and self-confidence.

Take a minute to answer this question: Do you remember how you were taught to write a paper in high school or in college?

Maybe you remember the five-paragraph essay, MLA formatting, or the RACE strategy, but were you ever taught specific strategies for planning and evaluating your papers?

While I was interviewing Dr. MacArthur about his recently completed IES project, he posed a similar question to me. He asked me how I navigated writing in college and if a teacher had ever explicitly taught me how to write. I realized that while I had some explicit teaching in text structure in high school, by the time I reached college, I relied heavily on feedback to inform my future writing. The idea that students learn from revising is a common view in writing education. However, this view does not always consider students who struggle with writing and who may need more explicit instruction, even in college.

As a teacher of high school students with learning disabilities, I often find that by the time many of my students reach my classroom, they feel defeated by the writing process. Writing is something that has become a source of fear and dread for them. My goal with each student is to find and develop strategies that bolster their writing skills and change writing from something that seems unattainable to something that they can do independently. I was excited to talk to Dr. MacArthur and learn more about the research that he and his have been doing. Below are his responses to the questions I posed.

What are the key components of the SSW curriculum?

The emphasis of SSW is to enable students to take control of their own learning through rhetorical analysis of genre. To do that, students are taught explicit strategies and cognitive procedures based on what good writers do. This is reinforced with metacognitive strategies that help students become aware of why they are using specific writing strategies and procedures and recognize how and when to transfer them to other classes. SSW places emphasis on genre-based strategies not only in the text but also in the planning and evaluation phases.

The heart of strategy instruction in SSW is the “think-aloud,” which is when instructors share, in real time, the thoughts that they are experiencing as they’re writing or editing a text to show how they are figuring things out. Instructors need to show—not just explain—how to write. What we writing instructors are teaching is invisible, so the think aloud makes the process visible to students. It also lets students see that writing is hard even for their teacher. Teachers can get stuck and need to work through it based on the strategies that are being taught.

What is the number one thing that you would tell a developmental or first-year writing teacher?

Teaching strategies to students on planning and evaluating their work helps improve writing. There have been hundreds of studies from K-12 (see these meta-analyses as examples 1, 2, 3) that show how strategy instruction works to improve writing. This experimental study of SSW adds to that literature and shows that strategic instruction with genre pedagogy can work in the postsecondary developmental writing environment.

What type of future research would you like to see done with the SSW curriculum?

There is a wealth of valuable research that could be done in the future. Future research could delve into how to build on the developmental course’s gain in subsequent courses. For example, it would be interesting to look at the transition between developmental writing courses and first-year composition in terms of pedagogical integration.

Another area of transfer is between compositions courses and disciplinary writing in postsecondary settings. For example, how could postsecondary institutions improve writing across the curriculum? How could strategy instruction similar to SSW work in this setting?

Additionally, strategy instruction started in special education, but it was found to be useful throughout the entire K-12 population. Similarly, SSW was found to be successful in developmental writing classrooms. It would be great to see the effects of SSW in first year composition classes.

You can find publications from this project and the earlier SSW project in ERIC here and here respectively. The What Works Clearinghouse also reviewed an earlier evaluation of the SSW here.

This blog was written by Rachael Higham, a graduate intern through the Virtual Student Federal Service Internships program, and facilitated by Dr. Meredith Larson (Meredith.Larson@ed.gov), a research analyst and program officer at NCER.

We hope you enjoyed the first NCER network lead spotlight! Today, we would like to introduce Dr. Thomas Brock, director of the Community College Research Center. Dr. Brock’s network, the Accelerating Recovery in Community Colleges (ARCC) Network, aims to provide timely, actionable research from the pandemic that policymakers and practitioners can use to help community colleges recover from the challenges introduced by the COVID-19 pandemic. Happy reading!

NCER: What are the mission and goals of the Accelerating Recovery in Community Colleges Network?

Dr. Brock: The primary goal of the ARCC Network is to provide timely, actionable research that policymakers and practitioners can use to help community colleges recover from the challenges introduced by the COVID-19 pandemic. These include steep drops in enrollment—particularly for students of color and male students—and learning losses associated with illness, stress, and challenges of online learning.

NCER: Why is the ARCC Network important to you?

Dr. Brock: The ARCC Network is important because community colleges are important. They enroll about one-third of all undergraduate college students in the U.S., including many who are from low-income backgrounds and the first in their families to attend college. The nation needs strong community colleges to help students advance educationally and economically. The nation also needs community colleges to prepare workers and support the economy in essential fields such as health care, information technology, construction trades, and manufacturing.

NCER: I understand that you had a central role in establishing the research networks grant program at IES. What is your view of a research network, and how does it differ from a traditional education research project?

Dr. Brock: There is an old adage that the whole is greater than the sum of its parts. With the research networks, IES intends to generate a body of work on a critical education problem or issue that is more impactful than an individual research project is likely to generate. This is because members of a research network come together regularly to discuss their ideas, tackle common methodological challenges, share data collection tools, and make sense of their emerging findings. They think about how to distill, align, and communicate research results from the early stages rather than as an afterthought. This benefits policymakers and practitioners, who look to researchers for insights and guidance. It also benefits the research community by building consensus on what has been learned and what new questions need to be addressed.

NCER: How do you think the ARCC Network will impact our nation’s community colleges?

Dr. Brock: Our hope is that the ARCC Network will help policymakers to be attentive to the needs of community colleges and shed light on the populations and places that need the most help. We also hope that the network will help identify promising policies and practices to promote rapid recovery.

NCER: What are some of the biggest challenges to recovery in community colleges?

Dr. Brock: Community colleges are largely funded based on enrollment. To date, the decline in enrollment has not led to too much reduction in academic programs or services because of the Higher Education Emergency Relief Fund (HEERF), authorized by Congress. HEERF funding ends after the 2022–23 academic year. If enrollments do not rebound quickly, community colleges will have to make significant cuts. This could lead to a downward spiral in which even fewer students enroll or persist, because they do not find the courses or services that they need.

Another challenge to recovery is learning loss. We know from the National Assessment of Educational Progress that there have been declines in reading and math achievement in K–12 schools during the pandemic. As these students mature and enter postsecondary education, they may be less well prepared for college-level work. Community colleges have made significant reforms to developmental education programs in recent years but will need to do more to ensure entering students succeed in college-level courses and make progress toward their academic and career goals.

Finally, we know that the pandemic has taken a severe toll on physical and mental health. Community colleges will need to find ways to reduce stress and promote wellness for everyone in their campus community–students, faculty, and staff.

NCER: What are some effective ways to translate education research into practice so that your work will have a direct impact on states and community college systems? What are some barriers to uptake of research outcomes by these organizations?

Dr. Brock: The ARCC Network will actively disseminate the research findings produced by individual research teams and by our national scan of community college enrollments and recovery practices. We will build a website that functions as an information hub for the most recent enrollment trends and reliable evidence on recovery strategies. We will conduct interactive workshops and webinars for state and local community college leaders and staff who are interested in learning from and adapting research-based practices to support pandemic recovery. We will use our connections with national organizations, like the American Association of Community Colleges and Achieving the Dream, and social media to ensure we reach a broad audience.

NCER: Are there some generalizable tools or lessons learned that are likely to come out of this network project that you think will benefit the education research community as a whole?

Dr. Brock: Yes. One area of focus for ARCC researchers, for example, is how to design and deliver effective online learning. Prior to the pandemic, most research on online learning in community colleges indicated it was not as effective as in-person instruction, but many colleges have upped their game with improved technology and better training and support for faculty who teach online. We have also seen from the pandemic that online learning benefits some students who might not otherwise attend community college, including students who live far from campus (especially in rural areas) or who are juggling demands of work and parenting. We hope to reframe the research debate so that it is less about online versus in-person instruction and more about how to provide online instruction most effectively to students who prefer this modality. We expect the lessons and tools from the ARCC Network will be broadly relevant to community colleges and may be adapted to other education sectors.

Thank you for reading our conversation with Dr. Thomas Brock! Come back tomorrow for our final grantee spotlight!  

In honor of National Disability Employment Awareness Month, we discussed NCSER-funded research on transition support for students with disabilities with principal investigators Meg Grigal and Clare Papay. Transition services prepare students for life after school and can include activities such as job training, post-secondary education, and support for independent living and community participation. This research team’s project, Moving Transition Forward: Exploration of College-Based and Conventional Transition Practices for Students with Intellectual Disability and Autism, examines outcomes for two transition approaches: a college-based transition and the conventional approach provided by most local education agencies. In the interview below, the researchers discuss recent results and how this information can improve the quality of transition services for students with disabilities.

What is the purpose of your project? What motivated you to conduct this research?

The bulk of existing transition research reflects knowledge about conventional transition services, 

or those services received by students with disabilities in high schools. An alternative approach, called college-based transition services, has been around for over 20 years, providing students with intellectual disability and autism a chance to experience college while continuing to receive support through special education. We wanted to explore and compare these two types of transition experiences and assess the outcomes for students. Using two existing datasets, our project conducted a series of interrelated analyses to look more closely at the transition services students with intellectual disability and/or autism (ID/A) are accessing and the association with youth outcomes in employment. Our hope is that our findings will contribute to the knowledge base on research-based college and career preparation for youth with ID/A.

Could you explain the difference between the two transition approaches (college-based and conventional) you are examining and how each prepares students for post-school life?

“Conventional transition services” is our way of describing the transition services typically provided to youth with disabilities across the United States. These services are documented in the data from the National Longitudinal Transition Study 2012 (NLTS 2012). College-based transition services, also known as dual enrollment or concurrent enrollment, provide students with intellectual disability access to college courses, internships, and employment and other campus activities during their final 2 to 3 years of secondary education. These experiences enable students to participate in career planning with a person-centered planning approach, enroll in college classes for educational and personal enrichment, engage in social activities alongside their college peers, and participate in community-based, paid work experiences that align with their employment goals.

What do the results from your research say about the employment outcomes and other transition outcomes of students with intellectual disability and autism participating in these transition programs?

To be blunt, our findings tell us that conventional transition services are not supporting students with ID/A to become employed after high school. We found a very low prevalence of school-based predictors of post-school success for students receiving conventional transition services. As an example, in our analysis of data from NLTS 2012, we found only 32% of youth with ID/A had paid employment in the previous 12 months. Paid employment in high school is a strong predictor of post-school employment. Additionally, there was low prevalence of other critical transition activities, including self-determination/self-advocacy, self-care/independent living skills, occupational courses, and work-study. Our findings highlight points of stagnation in access to college and career preparation for students with ID/A. Past low engagement rates in college preparation activities may have been attributed to the limited access youth with ID/A have had to positive employment outcomes and poor access to postsecondary education.

On a more promising note, when we look at data on students with ID/A who are enrolled in college-based transition programs, the picture is much brighter. We’ve found moderate to high prevalence of activities reflecting important predictors of post-school success (including­ paid employment while in high school, interagency collaboration, and learning skills in community settings). Students in college-based transition programs are enrolling in courses for college credit and taking courses to help them prepare for careers. These students are leaving K-12 education in a much better position to successfully be employed after high school than many of their peers who are receiving conventional transition services.

Based on what you have learned, what are the implications for practice and policy?

With increased access and opportunities to pursue further education after high school, youth with ID/A need college preparation activities to be a part of their standard education experience. Our findings suggest college-based transition services offer an approach that addresses both employment and college preparation. However, the availability of college-based transition programs depends upon whether school districts have established partnerships with a college or university. Greater availability of college-based transition services would provide the field with a better understanding of the essential elements of practice and associated outcomes of this approach. Our findings also show the need for substantial improvement in the access to college and career preparation for youth with ID/A in conventional transition services. Finally, these studies highlight the need for additional and more robust data in federal data systems reflecting information about the transition experiences of students with intellectual disability, autism, and other developmental disabilities. We need to know what their experiences between age 18-22 look like, how inclusive these experiences are, and what outcomes they achieve after they leave K-12 education.

How can families find more information regarding college-based transition programs in their area?

We are glad you asked! The Think College website has a College Search feature that includes all the college and university programs enrolling students with ID/A in the United States, including those who are working with transitioning youth. This is a great way for families to explore local options. When options don't exist, we encourage families to speak with their school administrators to work on developing partnerships with local colleges or universities. Think College has many resources about college-based transition available on our website. Additionally, our national help desk is always available to answer questions or offer help to those seeking information about inclusive higher education and college-based transition services. Send us questions at thinkcollegeta@gmail.com

Many thanks to Drs. Grigal and Papay for sharing their work with our readers! If you want to learn more about this project, including the results of their research, please visit the following website: https://thinkcollege.net/projects/mtf.

Meg Grigal is a senior research fellow at the Institute for Community Inclusion at the University of Massachusetts, Boston. At the Institute, she is co-director of Think College, a national organization focused on research, policy, and practice in inclusive higher education. Clare Papay is a senior research associate at the Institute for Community Inclusion.

This blog was produced by Shanna Bodenhamer, virtual student federal service intern at IES and graduate student at Texas A&M University, and Akilah Nelson, program officer for NCSER’s Transition to Postsecondary Education, Career, and/or Independent Living program.

Dual enrollment improves student college going and postsecondary success, but practitioners need help in understanding the impact of dual enrollment and in learning strategies associated with effective and equitable implementation. Under the auspices of the IES-funded Evaluation of Career and College Promise (CCP) project, the North Carolina Community College System suggested hosting a conference to build knowledge and capacity in the field about dual enrollment. The Evaluation of CCP is a partnership with the SERVE Center at the University of North Carolina at Greensboro, the North Carolina Department of Public Instruction, the North Carolina Community College System, and the RAND Corporation. In addition to the research goals—which involve looking at the implementation, impact, and cost of North Carolina’s dual enrollment program—the project also has a goal of capacity development for the agencies and for practitioners. As part of meeting this last goal, the project recently hosted a conference on Dual Enrollment: Accelerating Educational Attainment (DE21) with over 1,000 registrants from North Carolina and around the country.      

Julie Edmunds, the project’s principal investigator, discusses the DE21 conference.

Why host a conference on dual enrollment?

This was the brainchild of our partners at the North Carolina Community College System. They wanted to create an opportunity where researchers and practitioners could gather and share lessons learned from their respective work. The NC Community College System expected that we would be learning a lot from our project that we would want to share; they also knew that the people in the trenches had many valuable insights to help bridge the gap between research and practice. Because existing research shows that not all groups of students have the same access to dual enrollment, the project team decided collectively that the conference should have a strong focus on equity and to use the conference as a way to communicate and discuss strategies to support equity.

What happened at the conference?

We had a total of 40 sessions across two full days. There were dynamic keynote speakers, including Karen Stout from Achieving the Dream, and panels that discussed dual enrollment from the policy, research, student and parent perspectives. Although there was a strong North Carolina focus, there were sessions from other states such as Massachusetts, Texas, Indiana, and Ohio.

Conference presentations were organized into five themes: expanding access and equity, fostering college attainment, ensuring a successful transition to college and careers, preparing students for dual enrollment, and supporting success in dual enrollment courses.

The CCP study team presented findings from our evaluation of North Carolina’s dual enrollment pathways. We looked at individual and school-level factors associated with dual enrollment participation, such as student demographics, school size, locale, percentage of students from underrepresented minority groups, academic achievement, and workforce-orientation of students. Student socioeconomic level did not affect participation in dual enrollment. We also presented preliminary impacts of North Carolina’s three different dual enrollment pathways (college transfer, Career and Technical Education, and Cooperative Innovative High Schools or early colleges). Results from these three pathways showed that CCP participants had better high school outcomes such as higher school graduation rates and were more likely to enroll in postsecondary education. In addition, there were multiple sessions sharing research results from other states.

There were many presentations from practitioners that focused on topics like rigorous instruction, advising, participation of students with disabilities, creating strong secondary-postsecondary partnerships, using high school teachers as college instructors, among others. I need to give a huge shoutout to Katie Bao from the NC Community College System, who shepherded us all through the conference planning and implementation process.

What was the impact of the pandemic?

When we originally planned for the conference, we thought it would be in person. After the pandemic hit, we decided (as many other organizations did) to host it virtually. This made the conference much more accessible to a national audience, and we had participants and presenters from around the country.

What if someone missed the conference?

Another benefit of a virtual conference is that we are able to share all the sessions from the meeting. Please visit our site on YouTube to listen to the conference. 

What comes next?

Our study work continues, and we will share the results in a variety of ways, including through briefs and journal articles. We are also planning to host a second conference in 2023 and expect that it will have a virtual component so that it can continue to be available to a national audience.

Dr. Julie Edmunds is a Program Director at the SERVE Center at the University of North Carolina at Greensboro. In addition to being the PI on the Evaluation of Career and College Promise, she is one of the leading researchers on early college, a model that combines high school and college.