Manuela Nayantara Jeyaraj, a PhD student and researcher at the The Applied Intelligence Research Centre (AIRC) within the Technological University Dublin speaks with the University of Pittsburgh’s Health and Explainable AI podcast producer, Brent Phillips about explainability at the inference layer.

In this pilot episode of The Inference Layer, Manuela discusses her award-winning work on identifying cognitive bias in language models. She explains that while explicit bias is well-studied, her research focuses on implicit, subtle "cognitive biases" that models learn from human patterns, such as gender stereotypes in job recruitment or political descriptions. To address this, Manuela developed an algorithm that combines model-agnostic and model-specific explainability approaches to provide high-confidence justifications for AI decisions. She also highlights the creation of a massive, modern lexicon that captures gendered associations across a wide range of English, from archaic terms to contemporary slang found on TikTok and Instagram.

The conversation delves into the technical challenges of maintaining explainability at the inference layer, particularly when transitioning from high-compute cloud environments to resource-constrained edge devices like phones or wearables. Manuela emphasizes that for real-time applications clinical decision-making, explainability cannot be an "afterthought" and must be lightweight enough to run locally to ensure user privacy and trust. 

In the interview, Manuela highlights two ambitious areas she is eager to explore that connect the technical and human sides of AI. First, she is interested in developing high-confidence, real-time explainability for streaming data, where decisions must be justified in milliseconds without slowing down the model. This includes providing "counterfactual" explanations—identifying exactly what would need to change for a different outcome to occur, such as a patient's risk level shifting from high to low. Second, she wants to tackle the "storytelling" aspect of explainable AI (XAI), creating systems that can tailor the complexity and detail of an explanation to different stakeholders. For instance, in a recruitment scenario, she envisions a model that provides a deep technical justification for a recruiter while offering a more abstracted, helpful level of feedback for the job applicant.

The Inference Layer podcast is a collaborative initiative linking university AI labs, researchers, and supporting partners to explore the complexities of moving models from training to real-world deployment. Managed by volunteers, the series focuses on the intricate systems, chips, and stacks that define the inference layer. By highlighting advanced research and frontier challenges the podcast provides a platform for experts to discuss the cutting-edge developments driving the future of AI.