The fastest path to production AI isn't perfect architecture, according to Alexander Page. It's customer validation. In his former role of Principal AI Architect at BigPanda, he transformed an LLM-based prototype into "Biggy," an AI system for critical incident management. BigPanda moved beyond basic semantic search to build agentic integrations with ServiceNow and Jira, creating AI that understands organizational context and learns from incident history while helping with the entire lifecycle from detection through post-incident documentation.

Alexander also gives Ravin BigPanda's framework for measuring AI agent performance when traditional accuracy metrics fall short: combine user feedback with visibility into agent decision-making, allowing operators to drag-and-drop incorrect tool calls or sequence errors. He reveals how they encode this feedback into vector databases that influence future agent behavior, creating systems that genuinely improve over time. 

Topics discussed:

Listen to more episodes: 

Apple 

Spotify 

YouTube

Website