Agents of Change

How do you debug a system that doesn't give the same answer twice? Naman Goyal, Machine Learning Engineer at Google DeepMind, dives into the unique challenges of evaluating autonomous AI agents — where prompt context, model weights, and tool outputs all shift the outcome. Using a live demo built on a local Qwen model, he shows why evaluating an agent's full trajectory (not just its final answer) matters, how compression can cut token costs, and how "online judges" catch problems in real time. The demo agent under test: one built to investigate live incidents — including its own adversarial prompt injection attempts.

Mature AI organizations now run 40 to 80 nonhuman identities for every human employee — and most companies have no idea. Chad McDonald, CISO and Operating Advisor at TA Associates, explains why one-time authentication isn't enough for AI agents that need continuous verification, and introduces the CLAW framework — Control, Logging, Authentication, Watch — for managing AI identity at scale. The core message: this is classic security thinking, applied faster and bigger, in a world where AI is increasingly used to attack AI.

Finding product-market fit for AI is rarely a straight line. In this panel, Ketan Babaria (Chief Digital and AI Officer, eHealth), Beverly Rich Ingersoll (VP of AI and Innovation, UnitedLex), and Nitin Shingate (CTO, GoodRx) trade war stories from the health insurance and legal industries — from using Voice AI to cut customer wait times, to building AI advisory services for clients who don't yet trust the technology. They dig into change management risk, regulatory hurdles, and how to win organizational buy-in, before opening it up for audience Q&A on success metrics and the build-versus-buy debate.

What happens when an AI agent deletes your emails against direct instructions — or writes a hit piece on a human reviewer? Sunondo Ghosh, Chief Technology Officer at 2Q.ai, walks through real incidents where autonomous agents went off the rails, including Anthropic's own research on agents resorting to blackmail under conflicting goals. He introduces a four-pillar framework — constrain, verify, observe, interrupt — for keeping AI agents in check, and shares practical steps for auditing permissions, verifying outputs, and building in kill switches before things go wrong at scale.

AI is moving from giving advice to taking action — and that shift changes everything about how we monitor it. Amit Paka, COO and Founder of Fiddler AI, explains why traditional software monitoring can't keep up with autonomous agents, and what a real "control plane" for AI looks like. He walks through how Fiddler detects hallucinations, PII leakage, and other risks in real time, runs securely on customer premises, and scales cost-effectively across use cases from customer service to coding agents. The episode closes with a live Q&A on adoption timing and what enterprises should watch for next.

What happens when AI takes on one of healthcare’s biggest challenges—supply chain disruptions? Archie Mayani, Chief Product Officer at GHX (Global Healthcare Exchange), reveals how AI is transforming healthcare by predicting supply shortages, recommending alternatives, and ensuring patients get the care they need.

In this first episode of Agents of Change, we sit down with Ash Dhupar, Chief Data & Analytics Officer at Analog Devices, to unpack the real-world challenges of bringing AI—and especially agentic AI—out of the lab and into business at scale.