Keith Richman sits down with Hamel Husain, machine learning engineer and founder of Parlance Labs, to demystify AI evaluations (evals). Hamel breaks down why generic AI testing metrics fall short and how businesses can actually measure, debug, and improve their AI applications in the real world. They explore the pitfalls of simply slapping a chatbot on an existing product, the importance of iterative error analysis, and why starting simple with the most powerful models beats reaching for immediate complexity. Whether you're an executive fielding AI mandates or a developer building the stack, Hamel shares actionable advice on how to stop building the wrong things faster and start deploying AI that truly moves the needle.

• 00:00:00 Introduction: Why AI Testing Matters More Than You Think
• 00:01:52 What Are AI Evals and Why Every Business Needs Them
• 00:04:03 The Generic Metrics Trap: Why Off-the-Shelf Testing Fails
• 00:11:57 The Two Biggest Failure Modes in AI Implementation
• 00:13:37 Moving Fast vs Being Deliberate
• 00:15:04 The Slop Problem
• 00:21:24 Guardrails Done Right
• 00:23:45 Model Selection Strategy
• 00:27:25 Build vs Buy: When to Use Consulting vs Internal Teams
• 00:29:40 The Bootcamp Approach
• 00:31:19 The Million Lines of Code Myth
• 00:32:46 Embracing Mistakes and the Experimental Mindset
• 00:34:33 Personal Tech Stack and the OpenClaw Reality Check

#ArtificialIntelligence #MachineLearning #AITesting #TechLeadership #SoftwareEngineering #DataScience #OpenAI #ProductManagement #GenerativeAI #AIEvals

Sergio Furio, CEO of Creditas, joins Keith Richman to discuss how AI is fundamentally transforming his Brazilian fintech company. In this episode of Applied Intelligence, Sergio reveals how transitioning from traditional apps to 24/7 AI conversational agents is revolutionizing the customer experience and loan underwriting. He details how Creditas automated 85 complex legal risk assessments for collateralized lending and drastically boosted employee productivity—allowing the company to grow 30% annually while halving its overall headcount. Sergio also shares how AI platforms like Claude, Perplexity, and Notion are shifting company culture, turning traditional managers into hands-on "builders" who can execute end-to-end. If you want to know what real-world AI deployment looks like at a massive scale, this is a must-listen.

• 00:00:00 Intro
• 00:08:51 The AI Factory: Cultural Transformation Over Technology
• 00:09:28 Personal AI Transformation: From Excel to AI-First Workflows
• 00:16:26 Employees as Air Traffic Controllers
• 00:23:07 Liquidity Risk and Intelligent Data Orchestration
• 00:28:04 Expanding Access: How AI Reduces Costs and Widens the Funnel
• 00:29:37 Perplexity for Proof of Concepts: Building Agents in Real-Time
• 00:32:29 Builders Over Managers

Keith Richman sits down with Jeff McMillan, former Head of AI at Morgan Stanley, to unpack the realities of deploying enterprise AI. Jeff shares his journey of partnering with OpenAI in early 2022 and breaks down his six-layer framework for scaling AI effectively—emphasizing why models alone aren't enough. They explore the critical importance of data quality, governance, and the emerging "agentic orchestration" layer that will define the future of corporate AI. Jeff also explains why he believes the "Head of AI" title should be a temporary role and why giving every employee access to AI tools is the best way to drive adoption and reduce risk. If you're a business leader navigating AI integration, this episode delivers practical, battle-tested strategies.

• 00:00:00 Introduction: From Morgan Stanley to AI Transformation
• 00:00:41 The Unlikely Path: From Army Lieutenant to AI Leader
• 00:03:33 The OpenAI Partnership: First Enterprise Customer
• 00:06:47 The Six-Layer Enterprise AI Stack
• 00:08:45 Why Data Quality Still Matters More Than Models
• 00:09:58 The Agentic Orchestration Layer: The Future of Enterprise AI
• 00:14:56 MCP Protocol: The Railway Gauge for AI
• 00:17:22 Build vs Buy: Strategic Decisions for Different Company Sizes
• 00:23:44 The Ferrari in the Walmart Parking Lot Problem
• 00:28:13 Measuring AI Success: From Contact Velocity to Revenue
• 00:32:34 The Three-Year Timeline: Realistic Expectations for AI Transformation
• 00:36:32 Shadow AI and Enterprise Tool Strategy

#ArtificialIntelligence #EnterpriseAI #BusinessLeadership #MachineLearning #TechInnovation #DataAnalytics #GenerativeAI #FutureOfWork #AIStrategy #AgenticAI

Is AI actually helping students learn? In this episode of Applied Intelligence, host Keith Richman sits down with Jason Robinovitz, an education business leader and micro-school founder, to discuss the real-world impact of AI in education. Jason shares why his school prioritizes human connection over screens, how he practically integrates tools like ChatGPT and Claude to empower teachers, and why heavily marketed "two-hour AI schools" might leave kids unprepared for the real world. They also dive into the ethics of AI in college applications and why AI might actually widen the educational gap rather than close it. Whether you're an educator, business owner, or parent navigating technology in the classroom, this conversation offers grounded, actionable insights on deploying AI without losing the human element.

Timestamp:

• 00:00:00 Intro
• 00:00:54 The Micro-School Model
• 00:01:40 The ChatGPT Moment
• 00:03:46 Detecting AI Usage
• 00:06:16 Maximizing Human Connection Time
• 00:07:16 The Alpha School Debate
• 00:11:24 Ethical AI Usage
• 00:12:42 The English Major's Revenge
• 00:15:30 The Future of Tutoring
• 00:17:53 Why AI May Increase Educational Inequality

#AI #Education #EdTech #ArtificialIntelligence #Tutoring #Business #Technology #FutureOfEducation #AppliedIntelligence

Brendan Falk joins Keith to dismantle the hype around enterprise AI. While AI can generate software instantly, Falk explains why legacy integrations and edge cases make real-world deployment a massive challenge. He shares his proven framework for prioritizing AI initiatives based on business value versus technical feasibility and offers actionable strategies for upskilling teams. Whether you are evaluating internal tools, looking to automate customer support, or trying to bridge the gap between AI capabilities and actual ROI, this episode provides a grounded roadmap for success.

#artificialintelligence #businessstrategy #enterpriseai #nocode #startups #softwareengineering #aws #productivity #techtrends #management

Numan CEO Sokratis Papafloratos joins Keith to discuss the delicate balance of deploying AI in a highly regulated healthcare environment. Sokratis shares his unique perspective, having studied neural networks 25 years ago before building a digital health platform that has served 800,000 patients. They dive into why Numan uses a hybrid AI-human model for behavioral coaching while strictly avoiding AI for clinical diagnostics due to safety concerns. Sokratis details their internal tech stack—including Cursor for coding and Intercom’s Fin for support—and explains the critical process of converting unstructured patient data into actionable insights. This is a pragmatic look at how to scale healthcare safely using artificial intelligence.

• 00:00:00 Introduction: AI's Promise for Healthcare Access
• 00:00:38 From Neural Networks to Numan: A 25-Year AI Journey
• 00:06:11 The ChatGPT Watershed Moment and Healthcare Caution
• 00:10:56 Hybrid AI-Human Coaching: The Current Model
• 00:17:32 Democratizing Healthcare Through AI Scale
• 00:22:52 AI Across the Organization: Coding, Customer Service, and Beyond
• 00:24:09 Developer Productivity
• 00:43:11 Security, Data Privacy, and the Healthcare AI Guardrails
• 00:46:44 Personalized Healthcare Journeys

#AI #Healthcare #DigitalHealth #Business #Technology #Startups #MachineLearning #Leadership #Innovation #MedTech

Voi CEO Fredrik Hjelm joins Keith Richman to reveal how the European micro-mobility giant is transitioning into an AI-native organization. Fredrik breaks down his aggressive "build vs. buy" strategy, explaining how Voi reduced legacy SaaS spend by nearly 40% by replacing software with custom LLM tools. The conversation covers the evolution from basic chatbots to autonomous agents, the reality of "Shadow AI," and why AI curiosity is now a hiring requirement. From automating tender bids to replacing junior engineering tasks, this is a masterclass in practical, high-impact AI deployment for operators.

#AI #ArtificialIntelligence #BusinessStrategy #CEO #TechTrends #SaaS #Voi #Automation #Leadership #MachineLearning

• 00:00:00 Introduction: Hiring for AI Readiness
• 00:00:30 Voice's Journey: From Hypergrowth to AI-First
• 00:01:14 The Klarna Moment: Early AI Inspiration
• 00:05:10 Phase One: The Year of the Chatbot (2023)
• 00:05:43 Organizational Change Management for AI
• 00:09:04 Shadow AI and the White-Listing Process
• 00:11:12 Phase Two: Coding Assistants and Vertical AI Tools
• 00:13:36 Buy vs Build: Rethinking the SaaS Stack
• 00:16:47 Mapping Software Products for AI Replacement
• 00:18:02 Removing Human from the Loop
• 00:19:01 Top-Down Cross-Functional AI Implementation
• 00:21:24 The New Software Sales Reality
• 00:23:50 Cost Savings vs Revenue Growth with AI
• 00:27:38 The Rise of 100x Engineers
• 00:30:01 Hiring for AI Aptitude Across All Roles
• 00:31:13 Domain-Specific AI Tool Stacks
• 00:32:28 Structured vs Unstructured Data: The AI Challenge
• 00:35:22 Understanding LLMs in Enterprise Settings
• 00:37:07 Case Study: Liquidize Marketing Automation
• 00:38:50 LLM Orchestration and Model Selection
• 00:40:34 Becoming the World's First AI-Native Micromobility Platform
• 00:42:13 Constraints: The Key to Safe AI Automation
• 00:44:05 Organizational Response: From Resistance to Enthusiasm
• 00:46:54 Creating the Head of AI Role
• 00:50:42 The Results: Double Revenue, Half the Staff
• 00:51:41 Frederik's Personal AI Tech Stack
• 00:56:22 The Biggest Win: Company-Wide AI Adoption

Anthony Soohoo, CEO of MoneyGram, joins host Keith Richman to discuss how he is transforming an 80-year-old legacy payments company into an AI-first organization. Anthony shares his practical philosophy on AI adoption: start with a clear problem statement rather than the technology itself, and manually validate workflows before automating them. He explains why enterprise-wide goals must supersede departmental silos and predicts a massive disruption in the enterprise software market where costs drop by 80% while functionality increases tenfold. This episode is a masterclass for leaders looking to navigate the gap between AI hype and real-world operational implementation

#AI #BusinessStrategy #DigitalTransformation #FinTech #Leadership #ArtificialIntelligence #TechTrends #CEO #MoneyGram #AppliedIntelligence

• 00:00:00 Introduction: AI Transformation at MoneyGram
• 00:00:40 MoneyGram's Legacy Business and Digital Transformation
• 00:03:58 Building an AI-First Culture
• 00:05:56 Starting with Problem Statements, Not Functions
• 00:09:48 Communicating AI to Employees
• 00:11:30 Outcome-Based AI Strategy vs Functional Automation
• 00:14:27 The Future of Enterprise Software
• 00:15:44 Vetting AI Vendors and Partners
• 00:21:06 Enterprise Goals vs Departmental Goals
• 00:25:10 Data Strategy and Infrastructure
• 00:32:35 Early Wins and Productivity Gains
• 00:34:25 Hiring for AI Success: Enterprise Mindset
• 00:36:00 Data Labeling and Training Approaches
• 00:39:54 Personal AI Tech Stack
• 00:41:29 Advice for 100M Revenue Companies
• 00:42:24 The Transformation Opportunity