Software Engineering Institute (SEI) Podcast Series

AI components, whether traditional machine learning or generative AI, are embedded in many present-day systems. However, despite advances in automation, infrastructure, and tooling for developing AI components, many do not leave the prototype stage or reach production because they fail to meet overall system quality expectations. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Rachel Brower-Sinning and Robert Edman, both machine learning scientists in the SEI's Tactical Edge and AI-Enabled Systems Initiative, sit down with Grace Lewis, the initiative's lead and an SEI principal researcher, to discuss a quality model for machine learning components to support their proper testing and evaluation. The research team that developed the model also included Alex Derr, Sebastián Echeverría, and Ipek Ozkaya from the SEI, and, from Carnegie Mellon University, Kate Maffey. Colin Beaudoin, a PhD intern at the SEI in the summer of 2025 who is now an assistant professor at Fairfield University, also worked on the model.

The recent explosion in large language model (LLM) technology has highlighted the challenges of using public generative artificial intelligence tools in classified environments, especially for software analysis. Currently, software analysis falls on the shoulders of static analysis tools and manual code review, which tend to provide limited technical depth and are often time-consuming in practice. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Ryan Karl, an SEI embedded engineer, and John Robert, deputy director of the SEI's Software Solutions Division, discuss their work on using LLMs in unclassified environments to rapidly develop tools that accelerate software analysis in classified environments with improved accuracy for certain software analysis tasks.

Data breaches remind us of the importance of cyber resilience as an essential element of survivability and continuity of operations for all organizations, especially those operating mission-essential systems, high-value systems, and/or critical assets. Resilience is also critical to reducing the costs associated with security breaches as well as minimizing damage to mission-essential systems caused by adverse events. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Patsy Bulisco, Analysis Team Lead in the SEI's CERT Division, sits down with Timothy Morrow, Situational Awareness Technical Manager, also in CERT, to discuss an approach to using data analytics as a "force multiplier" for cyber resilience and suggest best practices to help organizations gain situational awareness on their current security posture.

Software-defined warfare is today's reality for national security, shifting the emphasis in military operations from hardware to software. In the latest podcast from the Carnegie Mellon University Software Engineering Institute, SEI director Paul Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss the evolution of software-defined warfare and the ways in which software engineering practices can meaningfully address the challenges of implementing software on the battlefield.

In aviation, waypoints guide pilots through complex flight plans, providing some structure but maintaining flexibility. Kevin Dooley, a senior Agile transformation leader at the SEI, adopted this concept to solve one of defense acquisition's most persistent challenges: synchronizing dozens of interdependent teams without drowning in administrative noise and overhead. In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Dooley, who co-developed the Waypoints Framework with Air Force Major Adam Satterfield, sits down with Eileen Wrubel, SEI technical director for Smart Software Acquisition to discuss Waypoints and how it can help teams visualize their work and own processes so they can start collaborating.

Experimentation and validation of LLM performance is critical when building LLM-driven systems that must reliably deliver a service, from customer service chat bots to intelligence analysis tools. To help teams meet the need for rigorous evaluation methods, a research team in the SEI's AI Division led by Violet Turri has developed the Expanding Large Language Model Metrics (ELM) library, which is built on best practices for LLM evaluation and benchmarking. In the latest episode from the Carnegie Mellon University Software Engineering Institute Podcast Series, Turri sits down with Katie Robinson, a design researcher also in the SEI's AI division, to discuss the library, which turns evaluation from an ad-hoc process into a repeatable, extensible framework.

Data poisoning—where adversaries tamper with training data to corrupt model behavior—poses significant risks as AI adoption expands across critical sectors. Organizations without mechanisms in place to detect or prevent data poisoning are open to an avenue of attack that, once exploited, is difficult to remediate. Machine unlearning and model retraining are not always viable or effective solutions. In today's operational climate, where threat actors look to influence models and degrade the trust of users through incorrect behaviors, preventing data poisoning is more important than ever.

In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.

Data poisoning—where adversaries tamper with training data to corrupt model behavior—poses significant risks as AI adoption expands across critical sectors. Organizations without mechanisms in place to detect or prevent data poisoning are open to an avenue of attack that, once exploited, is difficult to remediate. Machine unlearning and model retraining are not always viable or effective solutions. In today's operational climate, where threat actors look to influence models and degrade the trust of users through incorrect behaviors, preventing data poisoning is more important than ever.

In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.

As recently as December 2025, the Carnegie Mellon University Software Engineering Institute (SEI's) CERT Coordination Center (CERT/CC) documented a UEFI-related vulnerability in certain motherboard models, illustrating that early-boot firmware behavior continues to present security challenges despite requiring local physical access to exploit. While CERT/CC reported seven UEFI vulnerabilities in 2025, that number remains small compared to reported vulnerabilities in other software. However, the consequences of a potential UEFI attack are often more serious given the extremely high privileges UEFI firmware possesses. In our latest SEI Podcast, Vijay Sarvepalli, a senior information security architect specializing in vulnerability and threat analysis in CERT, sits down with Michael Winter, deputy technical director of threat analysis in CERT, to discuss research and mitigation of UEFI vulnerabilities and discuss a new tool, the CERT UEFI parser, an open source tool that uses program analysis to reveal the architecture of UEFI software, and explore this veiled source of vulnerabilities.

In February 2026, Paul Nielsen announced that he will transition out of his role as director and chief executive officer of the Software Engineering Institute (SEI) at Carnegie Mellon University. During Nielsen's tenure, the SEI has marked major institutional milestones that underscore its enduring role in strengthening the security, resilience, and reliability of the nation's software- and AI-intensive systems. The institute recently celebrated 40 years of innovation and saw its contract renewed, which paved the way for CMU to operate the SEI for another five years. In our latest SEI podcast, Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss his legacy at the SEI, the impact of mentors, and the importance of encouraging scientists and engineers to do their best work.