Inside IES Research

Notes from NCER & NCSER

Graduate Student Reflections on Engaging Research Opportunities

Engaging students in research can enrich their knowledge and support their future confidence to pursue research careers. In this interview blog, Dr. Allen Ruby, Associate Commissioner for the Policy and Systems Division at NCER, asked four doctoral graduate students at Montclair State University, Melissa Escobar, Taylor Walls, Hannah Thomas, and Marline Francois, to reflect on what attracted them to an IES-funded research project led by Dr. Carrie Masia. The project aims to improve education outcomes for Black American high school students with anxiety attending urban public schools through the development of culturally-responsive interventions.

What are your research interests, and how does this project align with your interests?

Melissa Escobar (ME): My research interests focus on optimizing access to evidence-based treatments for racial minority youth by training frontline providers in community and primary care settings to deliver them. I distinctly remember when I decided to pursue a career in psychology. I was working at a community youth center when the struggles of a Latinx mother deeply impacted me. Her husband's deportation to Mexico significantly altered her son's mood and schoolwork. She tearfully confided in me about her difficulties accessing mental health services for her son. She struggled to find a qualified provider who she believed understood their family's concerns, and the high cost of services and transportation prevented her from seeking care. Seeing the combination of cultural and structural barriers that influence mental health disparities within marginalized groups, I now align myself with research that advocates for high-quality depression and anxiety treatments in accessible locations for minority youth. This is why I found Dr. Masia's project a perfect fit for me. The project links the behavioral health and education sector to improve the mental health and academic achievement of historically marginalized youth with impairing anxiety.

Taylor Walls (TW): My research interests center around developing, implementing, and evaluating culturally sensitive interventions for children and adolescents with internalizing disorders in schools. A primary goal of this project is to use a school-based group intervention that has been shown to be effective in reducing social anxiety and revise it to address the unique needs of Black American students. It aims to consider the context of urban public schools and the culture of Black American adolescents. I have read about the importance of cultural adaptations to improve the quality and availability of these interventions for racial and ethnic minorities, and I welcome the opportunity to work closely on a project like this firsthand.

 

Hannah Thomas (HT): My research interests include optimizing evidence-based interventions for children and adolescents and the role of risk and protective factors in the development of internalizing disorders. My interest in these areas began when I worked at a summer program focused on bringing high school-aged student athletes, often from underserved communities, to learn leadership and sport psychology skills. This was a transformative experience, ultimately solidifying my interest to work with youth and interventions that teach skills to handle adversity. I was drawn to this IES project because it provides the opportunity to work with youth in a meaningful and impactful way.

 

Marline Francois (MF): My research interest is in exploring the psychological well-being of adolescent Black girls that experience racial and gender discrimination in education spaces. Furthermore, I am interested in creating gender and race-specific interventions for Black adolescent girls. My interest began after spending 15 years working as a therapist and recognizing the lack of interventions specifically for Black youth. I was also affected by the lack of adequate mental health services for Black youth coupled with their alarming increase in suicide rates. What I love about this project is that it aligns with my research interests and directly involves adolescents, which provides opportunities to learn from them and have them share their lived experiences.

 

What has been the most exciting aspect of this project for you?

ME: The most attractive part has been the opportunity to hear directly from students and parents on how to best address the needs of racial-ethnic minority families and develop culturally sensitive assessments and treatment strategies. Fully engaging with the community makes the work more meaningful to me. Also, the chance to have a hands-on approach in the research process from collecting qualitative data to developing interview guides and coding schemes makes me feel like my contributions are making a difference.

TW: As a fourth-year graduate student, this project is particularly exciting for me because I will be analyzing a portion of the data for my dissertation. It has expanded my research skills to formulate my own research questions to contribute information that is novel and of interest to my field. Furthermore, I enjoyed the aspects of this project that mirror my work as a clinician — speaking with children and their parents in one-on-one and small group settings to hear about their experiences and their feedback on how this program may be better tailored to their community. Having conversations with the individuals we want to impact makes this work particularly meaningful.

HT: The most exciting aspect of this project is the ability to be involved in multiple roles. I started at the beginning of the summer, and so far, I have been involved in conducting focus groups and developing a coding scheme for interview transcriptions. This excites me because I am able to diversify my skills as a researcher and gain experience in various research methods that may be useful for my dissertation.

MF: As a 3rd-year doctoral candidate, it has been exciting to see the process of this project from the beginning and being able to interview and interact with students and parents. I enjoyed the recruitment process and conducting focus groups and individual interviews. As a qualitative researcher, I appreciate the hands-on experience of learning how to conduct and the in-depth experiences shared by the students and parents, which will aid us in creating a more culturally sensitive intervention for Black youth.

What do you look for in a research supervisor or mentor?

ME: In addition to similar research interests, I look for a mentor who is respectful of personal boundaries. As graduate students, we have a lot of different responsibilities and having a mentor that shows you how to balance those roles is vital in keeping students engaged and successful. I have been fortunate to have a mentor who ensures that all work is evenly spread across all research team members. I also appreciate mentors who considers their students' personal goals and finds opportunities that align with them. For example, at the beginning of my graduate program, my mentor asked me what my goals were for the year, the program, and what type of work setting I saw myself in after graduation. This conversation has been beneficial in finding research opportunities, grants/scholarships, and clinical experiences that will help me meet my goals outside of working on research. Lastly, I take into consideration how available the mentor is. It is essential to maintain good communication through regular meetings and/or emails. This way, regular communication and feedback can happen in appropriate time frames, and any issues that arise are resolved quickly.

TW: It's important to me that a mentor has both interest in and time to support me in fostering my competence as a research professional and advancing my short- and long-term goals. This can be demonstrated in a number of ways, including recommending academic and professional development activities that will build my skills, providing constructive feedback in a respectful and supportive manner, and helping me manage challenges as they arise. I also prefer that my mentor shares in my passion for child-focused research and is eager to connect me with collaborators for projects or networking. Over the years, I've learned that I work best with mentors who grant me autonomy in my work, but I also benefit from frequent check-ins and strict deadlines. Finally, I appreciate a research mentor that provides encouragement and flexibility and acknowledges the importance of self-care and well-being.

HT: I looked for a research supervisor/mentor whose research interests aligned with mine. I found this to be important because not only are they knowledgeable in the area of research that I am interested in, but they also provide the right research opportunities to develop me on the road to an independent career.

MF: It was important for me to find a mentor with a research background that aligns to my research interests. I wanted to have someone that cared about my research interests and professional growth. Furthermore, it was important for me to find someone who was not afraid to give me constructive criticism on my ideas but also assist me with strategically planning for my future. As someone who values balance, it is also important for me to have a mentor who values my well-being.

What challenges have you faced when trying to find research projects that appeal to you, and what feedback would you give to graduate programs or faculty to better engage students in research? 

ME:  As a first-generation student, I did not know how to navigate finding research opportunities or emailing professors about potential opportunities. I quickly learned that most research opportunities aren't advertised and finding a role on a research team usually comes from word of mouth. I would encourage programs and faculty to do more to advertise research opportunities. I would also recommend that faculty welcome the involvement of undergraduate students in their labs. My research career began during the sophomore year of my undergraduate education. I would not have the experience I have today if that professor had not given me a chance. Even if the roles are small like doing audio transcription or data entry, all experience is valuable. Another suggestion would be to create a mentorship model within the lab with more senior students mentoring newer students. When relationships like this are built, students may feel more comfortable to try out new roles in the lab.

TW: In order to find research projects that appeal to me, I am diligent about seeking out faculty who are already doing work that interests me and are open to bringing on a collaborator. I appreciated that my graduate master's program hosted an open house at the beginning of each semester where students could meet faculty and be oriented to the research labs that were available to them. What helped me first want to get engaged in research was meeting faculty who were outwardly passionate about their work and created unique avenues for their students to get involved.

HT: I feel that I have been fortunate in obtaining research projects that appeal to me, and I attribute that to aligning myself with the right mentors. My past and current mentors are collaborative with other faculty and labs, which have allowed me to participate in a variety of research opportunities, further refining my research interests.  My suggestion for graduate programs and faculty to better engage students in research is to encourage collaboration across faculty and labs. 

MF: When I initially started my doctoral program, it was challenging to find the right research project that aligned with my interests. However, I believe it is important as a doctoral student to be flexible and adaptable to learning from other mentors. Furthermore, I found myself connecting with scholars and students from other disciplines and universities that had similar interests as myself. This has allowed me to learn to properly advocate for my needs and find a research lab that was more suitable for me. The advice I would give to graduate programs is to create more opportunities for graduate students across departments and disciplines tailored to various research needs. I also believe that universities can create more professional development training opportunities to better engage students in research, such as how to apply for competitive fellowships and grant opportunities during their studies.  


This blog was produced by Allen Ruby (Allen.Ruby@ed.gov), Associate Commissioner for Policy and Systems Division, NCER. 

 

Six Strategies for Effectively Communicating Research Findings to Decision Makers

Researchers must possess the ability to clearly communicate research findings to non-technical audiences, including decision makers who may have limited time and varying levels of tolerance for data-rich reports. We and our colleagues recently honed these skills while preparing research briefs for the Virginia Department of Education (VDOE) as part of an IES-funded partnership project between VDOE and the University of Virginia exploring the impacts of the COVID-19 pandemic on students and teachers. These briefs are a key mechanism through which our project services the purpose of IES’s Using Longitudinal Data to Support State Education Policymaking Grantmaking Programs to generate useful findings to inform the decision making of policy makers and education leaders at the state, district, and school levels.

In their initial feedback, VDOE described the briefs as “too technical.” When we led with the numbers, our intended audience quickly became overwhelmed by the need to also interpret the findings on their own. Our conversations with VDOE provided helpful direction on how we could revise the briefs to better reach non-technical, decision-making audiences in Virginia and beyond. We share six strategies we have applied to all our research briefs.

  • Yes, briefs need a summary too: The draft briefs were short (4-7 pages) inclusive of figures and endnotes, and they began with a list of key findings. Based on the feedback, we morphed this list into a proper summary of the brief. Many of the decision makers we want to reach only have time to read a page summary, and that summary needs to be self-contained. Without additional context, the initial list of key findings would have had minimal impact.
  • Lead with the headline: Numbers are a powerful tool for storytelling; however, too many numbers can also be hard for many people—researchers and non-researchers alike—to consume. We therefore edited each paragraph to lead with a numbers-free sentence that provides the main take away from the analysis and followed that up with the supporting evidence (the numbers).
  • Answer the question: Our initial groundwork to develop solid relationships with agency staff allowed us to identify priority questions on which to focus the briefs. While several tangential but interesting findings also resulted from our analysis, the briefs we developed only focused on answering the priority research questions. Tangential findings can be explored in more depth in future research projects.
  • Accurate but not over-caveated: All research makes some assumptions and has some limitations. The average non-technical audience member is unlikely to want a thorough detailing of each of these; however, some are too important to exclude. We chose to include those that were most vital to helping the reader make the correct interpretation.
  • A picture speaks a thousand words: This was something at which our initial drafts succeeded. Rather than providing tables of statistics, we included simple, well-labeled figures that clearly presented the key findings graphically to visually tell the story.
  • Conclude by summarizing not extrapolating: The purpose of these briefs was to describe the changes that the pandemic wrought to Virginia’s public schools and convey that knowledge to decision makers charged with plotting a course forward. The briefs were not intended to provide explicit guidance or recommendations to those decision makers.

These strategies, of course, are also useful when writing for technical audiences. While their training and experiences may equip them to consume research that doesn’t exhibit these six strategies, using these strategies will enhance the impact of your research findings with even the most technical of audiences.


Luke C. Miller is a Research Associate Professor at the University of Virginia’s School of Education and Human Development. He is the lead researcher and co-Principal Investigator on the IES-funded project led by VDOE in partnership with UVA.

Jennifer Piver-Renna is the Director of the Office of Research in the Department of Data, Research and Technology at the Virginia Department of Education. She is the state education agency (SEA) co-Investigator on the IES-funded project.

This blog was produced by Allen Ruby (Allen.Ruby@ed.gov), Associate Commissioner for Policy and Systems Division, NCER.  

Google Acquires Intellectual Property for IES-Supported Education Technology Products Moby.Read and SkillCheck

On April 1, 2022, Google acquired the intellectual property (IP) rights for Moby.Read and SkillCheck, education technology products developed through IES programs by California-based Analytics Measures, Inc. (AMI). AMI will continue as a small business and is honoring school contracts that use Moby.Read and SkillCheck until 2024.

Moby.Read is a technology-delivered, fully automated, oral reading fluency (ORF) assessment that is self-administered by grade school students. As students read a passage aloud into a tablet, the speech-recognition software generates an assessment of ORF in real time through natural language processing software that analyzes text passages of the read-aloud performances. SkillCheck is a component of Moby.Read that employs natural language processing software to analyze recordings and produce interactive report pages that rate and illustrate the student's basic reading skills.

 

 

The technologies were developed over two decades with IES funding. Beginning in 2002, AMI designed several early prototypes to be used for ORFs as a part of the National Assessment of Educational Progress and other national assessments administered by IES’s National Center for Education Statistics. In 2016 and 2017, the IES Small Business Innovation Research program (ED/IES SBIR) funded AMI to develop and test Moby.Read to be used in schools at scale. With 2020 and 2021 ED/IES SBIR awards, AMI developed the SkillCheck as an additional component of Moby.Read to provide educators activities to inform instruction. AMI conducted research at key points over 20 years to validate the results of the assessment.

Since commercial launch in 2019, the Moby.Read and SkillCheck have been used for more than 30,000 student assessments in 30 states. Google acquired the Moby.Read and SkillCheck IP with plans to incorporate these tools into Google suite of products for education.

For additional information on the research, development, and commercialization of these technologies, see this Success Story on the ED/IES SBIR website.


Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences. Please contact Edward.Metz@ed.gov with questions or for more information.

 

Investing in Next Generation Technologies for Education and Special Education

The Department of Education’s (ED) Small Business Innovation Research (SBIR) program, administered by the Institute of Education Sciences (IES), funds entrepreneurial developers to create the next generation of technology products for students, teachers, and administrators in education and special education. The program, known as ED/IES SBIR, emphasizes an iterative design and development process and pilot research to test the feasibility, usability, and promise of new products to improve outcomes. The program also focuses on planning for commercialization so that the products can reach schools and end-users and be sustained over time.

In recent years, millions of students in tens of thousands of schools around the country have used technologies developed through ED/IES SBIR, including more than million students and teachers who used products for remote teaching and learning during the COVID-19 pandemic.

ED/IES SBIR Announces 2022 Awards

IES has made 10 2022 Phase I awards for $250,000*. During these 8 month projects, teams will develop and refine prototypes of new products and test their usability and initial feasibility. All awardees who complete a Phase I project will be eligible to apply for a Phase II award in 2023.

IES has made nine 2022 Phase II awards, which support further research and development of prototypes of education technology products that were developed under 2021 ED/IES SBIR Phase I awards. In these Phase II projects, teams will complete product development and conduct pilot studies in schools to demonstrate the usability and feasibility, fidelity of implementation, and the promise of the products to improve the intended outcomes.

IES also made one Direct to Phase II award to support the research, development, and evaluation of a new education technology product to ready an existing researcher-developed evidence-based intervention for use at scale and to plan for commercialization. The Direct to Phase II project is awarded without a prior Phase I award. All Phase II and the Direct to Phase II awards are for $1,000,000 for two-years. Across all awards, projects address different ages of students and content areas.

The list of all 2022 awards is posted here. This page will be updated with the two additional Phase I awards after the contracts are finalized.

 

 

The 2022 ED/IES SBIR awards highlight three trends that continue to emerge in the field of education technology.

Trend 1: Projects Are Employing Advanced Technologies to Personalize Learning and Generate Insights to Inform Tailored Instruction

About two-thirds of the new projects are developing software components that personalize teaching and learning, whether through artificial intelligence, machine learning, natural language processing, automated speech recognition, or algorithms. All these projects will include functionalities afforded by modern technology to personalize learning by adjusting content to the level of the individual learner, offer feedback and prompts to scaffold learning as students progress through the systems, and generate real-time actionable information for educators to track and understand student progress and adjust instruction accordingly. For example:

  • Charmtech Labs and Literably are fully developing reading assessments that provide feedback to inform instruction.
  • Sirius Thinking and studio:Sckaal are developing prototypes to formatively assess early grade school students in reading.
  • Sown To Grow and xSEL Labs are fully developing platforms to facilitate student social and emotional assessments and provide insights to educators.
  • Future Engineers is fully developing a platform for judges to provide feedback to students who enter STEM and educational challenges and contests.
  • Querium and 2Sigma School are developing prototypes to support math and computer science learning respectively.
  • ,Soterix is fully developing a smart walking cane and app for children with visual impairments to learn to navigate.
  • Alchemie is fully developing a product to provide audio cues to blind or visually impaired students learning science.
  • Star Autism Support is developing a prototype to support practitioners and parents of children with autism spectrum disorder.

Trend 2: Projects Focusing on Experiential and Hands-On Learning
Several new projects are combining hardware and software solutions to engage students through pedagogies employing game-based, hands-on, collaborative, or immersive learning:

  • Pocketlab is fully developing a matchbox-sized car with a sensor to collect physical science data as middle school students play.
  • GaiaXus is developing a prototype sensor used for environmental science field experiments.
  • Mind Trust is a developing a virtual reality escape room for biology learning.
  • Smart Girls is developing a prototype science game and accompanying real-world hands-on physical activity kits.
  • Indelible Learning is developing a prototype online multi-player game about the electoral college.
  • Edify is fully developing a school-based program for students to learn about, create, and play music.

Trend 3: Projects to Advance Research to Practice at Scale

Several new awards will advance existing education research-based practices into new technology products that are ready to be delivered at scale:

  • INSIGHTS is fully developing a new technology-delivered version to ready an NIH- and IES-supported social and emotional intervention for use at scale.
  • xSEL Laband Charmtech Labs (noted above) are building on prior IES-funded research-based interventions to create scalable products.
  • Scrible is developing an online writing platform in partnership with the National Writers Project based on prior Department of Education-funded research. 

 


*Note: Two additional 2022 Phase I awards are forthcoming in 2022. The contracts for these awards are delayed due to a back-up in the SAM registration process.

Stay tuned for updates on Twitter and Facebook as IES continues to support innovative forms of technology.

Edward Metz (Edward.Metz@ed.gov) is the Program Manager of the ED/IES SBIR program.

Michael Leonard (Michael.Leonard@ed.gov) is the Program Analyst of the ED/IES SBIR program.

 

Does Gifted Education Access Vary by District? A Study in Washington State

Students and their teacher work over a table with a large map on it.

States and localities have discretion over gifted programs, but surprisingly little large-scale research compares the education environments of students in gifted programs to high-achieving, non-gifted students or investigates how these learning environments vary across districts. In this guest blog, Ben Backes, James Cowan, and Dan Goldhaber discuss their IES-funded exploration study, where they  use administrative and survey data to describe the relationship between gifted participation and access to educational resources across nearly 300 school districts in Washington State.

Gifted Access and Participation in Washington

The underrepresentation of low-income and minority students in gifted programs has attracted attention because identification procedures often include nomination or referral processes requiring subjective evaluation of student ability. Nationally, low-income and non-White students are significantly less likely to participate in gifted programs. To better understand who is in these gifted programs in Washington State, we are investigating participation in gifted programs by student race/ethnicity and socioeconomic status in grades 4–12. Consistent with prior studies, relative to White students, we observe Asian students being more likely to be found in gifted programs, while Black, Hispanic, and free and reduced-price lunch students are less likely to receive gifted services. Washington districts frequently use universal screening policies, and the Black-White and Hispanic-White gifted gaps disappear once statistical adjustments for prior test scores are used. We find little association between use of modifications for underrepresented minorities or low-income students—as reported by district coordinators—and gifted participation.

In sum, we find consistent evidence of disparities in access to gifted programs conditional on student achievement in Washington for low-income students, but less consistent evidence of disparities by student race/ethnicity. However, we only observe data on student academic aptitude beginning in third grade, and many classification decisions are made before this time. There may be disparities in initial gifted classification decisions for younger students.

Unsurprisingly, participation in gifted programs does affect student learning environments. Gifted students are much more likely to sit in classrooms with other high-achieving students and in more homogenous classrooms. These differences persist even after limiting the sample to high achievers. These patterns are most pronounced in elementary school. Gifted students are taught by more qualified teachers in elementary and middle school, as measured by experience, licensure test scores, and educational attainment. However, these effects are very small.

Differences Across Districts and Program Types

We find that although gifted students do tend to take more advanced courses with higher-achieving peers, there is considerable variation in the design of gifted programming across school districts.

  • Although school districts tend to assign gifted students to more advanced academic tracks, we find that these effects are mostly concentrated in large urban and suburban districts. The estimated gifted effects on access to more advanced courses are typically much smaller in the western and eastern school districts in smaller cities and rural areas of the state.
  • Larger, higher income districts in cities and suburbs operate gifted programs that provide more significant changes in learning environments. Students in these programs are more likely to share classrooms with other gifted students and with high-achieving students, and—in the case of large districts—sit in smaller classrooms with more qualified teachers.
  • The structure of gifted programming also influences the type of instructional approaches districts employ. Self-contained gifted programs—where students are assigned to specialized classrooms for most of their instruction—report using a broad array of acceleration strategies. However, about one third of gifted students participate in programs offered through services in regular classrooms, where independent study, supplemental instruction, and flexible ability grouping appear to be important strategies.
  • Well under half of districts have established gifted curricula for math or ELA. About 20% of gifted students are districts that report having a districtwide math curriculum and 25% are in districts that report having districtwide ELA curriculum. This finding is consistent with another study that surveyed districts in three states.

What’s Next?

There is a growing body of empirical literature providing causal estimates of the effect of gifted participation on student achievement which generally uses administrative data from a single school district. The results from this study of gifted programs across an entire state suggest that district-specific gifted programming effects are likely to vary substantially as the nature of the programs vary substantially across districts. This implies both that we should be cautious about generalizing based on district-level studies and that the variation in findings across studies may be indicative of true variation in program effectiveness. In the next stage of this project, we plan to investigate the extent to which this heterogeneity generates differences in the relationship between gifted participation and student achievement.


Ben Backes is a Senior Economist with CALDER at the American Institutes for Research.

James Cowan is a Senior Researcher with CALDER at the American Institutes for Research.

Dan Goldhaber is the Director of CALDER at the American Institutes for Research and CEDR at the University of Washington.