Solving the Test Oracle Problem with Program Analysis and Generative AI

Abstract

Software impacts nearly every aspect of our lives, and software bugs are costly. Bugs can compromise system stability and security, and in extreme cases, can lead to loss of life—not to mention trillions of dollars in global economic losses. Testing is essential to detect these bugs early in the software development phase to ensure quality and reliability. Despite significant recent advancements, automated testing still faces critical challenges.

In this talk, I will present my research on addressing the test oracle problem—a fundamental challenge in software testing that directly impacts bug detection effectiveness. I will discuss how I have leveraged program analysis (PA) and generative artificial intelligence (GenAI) to assess and improve the adequacy, correctness, and strength of test oracles, thereby enhancing their effectiveness in identifying bugs. Finally, I will outline my vision for future research, focusing on combining PA and GenAI to advance automated testing, generate formal specifications, and address broader challenges in software quality assurance.

Biography

Soneya binta Hossain, PhD candidate in the Department of Computer Science at the University of Virginia, is advised by Professor Matthew Dwyer. Her research lies at the intersection of Software Engineering (SE), with a primary focus on Software Testing and the application of Generative AI (GenAI) in Software Engineering (GenAI4SE). Her work has been published in top-tier venues, including ICSE, FSE, and NeurIPS. She has gained industry experience through multiple internships at Amazon Web Services (AWS) and has received several prestigious honors, including the John A. Stankovic Outstanding Research Award and the Copenhaver Charitable Trust Bicentennial Fellowship.