Sign up to save your podcasts
Or
Share Just Now Possible
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Just Now Possible
Just Now Possible episodes
- From Prototype to Production: How Perk Built a Voice AI Agent That Makes 10,000 Calls a Week
Guests
- Steven Payne, Product Manager, Perk
- Gabriel Stock, Senior Engineering Manager, Perk
- Philipe Steiff, Senior Software Engineer, Perk
What we cover in this episode
- How Perk's team identified an AI use case by connecting prior experimentation with a real operational problem
- Why they chose Make.com for prototyping—and shipped to production without touching backend code
- The evolution from a single prompt to structured conversation stages (IVR handling, booking confirmation, payment request)
- How breaking up the agent's task dramatically improved reliability
- Building two eval systems: classification for success rates and LLM-as-judge for conversational behavior
- Why the team still listens to calls manually even with automated metrics
- The challenge of prompt engineering for voice: numbers, booking references, and text-to-speech markup
- Lessons learned from expanding to German (prompts in native language improve results)
- How this project uncovered other operational problems they didn't know existed
Resources & Links
- Perk
- Make.com – No-code automation platform used for the prototype
- Twilio – Voice/telephony provider
- 11 Labs – Text-to-speech provider (used in early experiments)
Chapters
00:00 Introduction to the Team
01:54 Understanding PERK's Mission02:59 Challenges in Travel Booking07:27 AI Solutions for Customer Care09:52 Prototyping with AI and Voice17:00 Implementing AI in Production25:51 Learning Through Trial and Error26:40 Prompting Challenges and Solutions27:58 Iterating on Prompts and Evaluations30:08 Scaling and Production Challenges32:43 Advanced Evaluation Techniques35:32 Real-World Applications and Success49:07 Future Directions and Expansion53:53 Conclusion and Team Reflections55 min - Building an AI Sleep Coach: How Rest is Making CBTI Principles Accessible to DIY Sleep Hackers
Guests
- Martin Siniawski, CEO and co-founder, Rest
- Ignacio, CTO, Rest
You'll hear how they:
- Discovered the sleep use case from podcast app user behavior (10% of users, but high willingness to pay)
- Used jobs-to-be-done research to identify "DIY sleep hackers" as an underserved segment
- Chose CBTI (Cognitive Behavioral Therapy for Insomnia) as their foundation—a clinically proven approach with 80% efficacy
- Evolved from text chatbot to voice-first AI using Vapi for voice and OpenAI for reasoning
- Built a memory system that remembers user context (like traveling, having a dog) with time-based relevance
- Created dynamic agendas that drive daily conversations based on sleep data, program stage, and user compliance
- Managed parallel development paths (text via OpenAI Assistants and voice via Vapi)
- Moved from massive system prompts to RAG for general sleep knowledge, keeping user data in prompts
- Navigated wellness vs. medical product positioning with clear guardrails against diagnosis and medication advice
- Used weekly error analysis with domain experts (sleep therapists) to drive product iterations
- Built LLM-powered evals for safety boundaries and experimented with Hamming for voice testing
Resources & Links
- Rest – AI sleep coach app
- Vapi – Voice agent platform Rest uses
- Langfuse – Observability and evals platform
- Hamming – Voice testing platform
- AI Evals Maven Course by Hamel Husain and Shreya Shankar (Get 35% off with Teresa's affiliate link)
Chapters00:00 Introduction to Rest and Its Founders
00:33 The Origin Story of the AI Sleep Coach02:07 Exploring the Podcast App and Sleep Use Case03:35 Transitioning to a Dedicated Sleep Audio App05:47 Understanding User Segments and Sleep Challenges07:45 Introduction to the AI Sleep Coach13:14 The Role of Voice in the AI Sleep Coach18:46 Daily User Interaction and Features21:30 Prototyping and Early Learnings28:09 Navigating Ethical and Regulatory Concerns30:39 Navigating the Line Between Health and Wellness Apps31:00 Incorporating Adjacent Disciplines into the App32:15 The Power of 24/7 Availability32:53 Evolution of the Chatbot and Error Analysis34:49 User Experience Improvements and Voice Integration46:49 Implementing Memory and Personalization50:18 Dynamic Agenda and User-Centric Conversations57:37 Evaluation and Guardrails01:00:05 Future Roadmap and Enhancements01:03:38 Combining Data Layers for Enhanced AI01:06:00 Conclusion and Final Thoughts1 hr 7 min - Turning Vendor Chaos into Answers: How Xelix Built an AI Helpdesk
Guests
- Claire Smid — AI Engineer, Xelix
- Emilija Gransaull — Back-End Tech Lead, Xelix
- Talal A. — Product Manager, Xelix
Key Takeaways
- Start narrow to win: pick high-volume, high-cost requests (invoice status & reminders).
- Enrichment > magic: accurate replies come from great retrieval/matching, not just a bigger LLM.
- Design for adoption: familiar inbox view helps onboarding, but a ticket-first UI unlocks AI features.
- Measure outcomes, not vibes: track “messages sent from Helpdesk”, % auto-resolved.
- Confidence builds trust: show match quality and response confidence so humans know when to edit.
0:00 Meet the Team: Claire, Emilija, and Talal
00:36 Introduction to Xelix and Its Products01:08 Understanding Accounts Payable Teams01:37 Help Desk Product Overview03:11 Challenges Faced by Accounts Payable Teams04:03 AI Integration in Help Desk05:47 Automating Reconciliation Requests07:45 Development Methodology: Carpaccio09:11 Prototyping and Beta Testing12:00 Manual Tagging and Data Collection16:39 Focusing on High-Impact Use Cases18:55 User Experience and Interface Design24:56 Pipeline Architecture and Email Processing28:21 Data Enrichment Challenges29:04 Handling Vendor Identification33:33 Email Thread Management36:15 Generating Accurate Responses40:48 Evaluating System Performance49:20 Future Developments and Goals54 min - When AI Becomes Your SRE: How Incident.io Is Automating Incident Response
Guests
- Lawrence Jones, Founding Engineer at Incident.io
- Ed Dean Product Lead for AI at Incident.io
Key Takeaways
- AI’s biggest impact comes from compressing time—identifying causes minutes instead of hours.
- Retrieval-augmented reasoning still benefits from simplicity: deterministic tagging and re-ranking often beat complex vector setups.
- Post-incident “time travel” evals let teams score AI accuracy after they know what really happened.
- Building trust in AI isn’t just about precision—it’s about showing reasoning and uncertainty in ways humans understand.
Mentioned Tools & Concepts
- Slack as the interface for human-AI collaboration
- PGVector and Postgres for retrieval experiments
- RAG (Retrieval-Augmented Generation)
- Multi-agent orchestration
- “AI as your company’s immune system”
Chapters
00:00 Meet the Founders: Lawrence and Ed00:41 Introduction to Incident.io01:25 Evolution of Incident.io Products02:14 Understanding SRE and Its Importance04:01 Real-World Incident Management05:51 The Role of AI in Incident Management10:12 Challenges and Innovations in AI SRE12:14 Prototyping and Iterating AI Solutions16:25 Refining Retrieval Strategies21:52 Balancing AI and Human Interaction32:06 User Experience and Trust in AI Systems36:08 Interactive Slack Integration37:08 Understanding the AI Investigation Process37:50 Parallel Checks and Data Sources38:35 Building Hypotheses and Refining Findings40:09 Human-Agent Collaboration49:23 Evaluating AI Effectivenessa01:04:13 Future Developments and Integrations1 hr 9 min - Building Trainline’s AI Travel Assistant: How a 25-Year-Old Company Went Agentic
Guests
- David Eason, Principal Product Manager at Trainline
- Billie Bradley, Product Manager, Travel Assistant at Trainline
- Matt Farrelly, Head of AI and Machine Learning at Trainline
Key Takeaways
- AI assistants need both scalable reasoning and deep domain context to be useful.
- Tool design and guardrails are as critical as prompt design in agent systems.
- LLM-as-judge evals make it possible to measure open-ended systems without massive labeling costs.
- Even legacy companies can move fast when they embrace experimentation and tight PM–engineering collaboration.
Chapters:
00:00 Introduction and Team Introductions00:51 Overview of Trainline's Mission and History02:30 AI Integration in Trainline's Services05:08 Challenges and Solutions in AI Implementation06:52 Building and Iterating the AI Travel Assistant14:58 User Experience and Guardrails22:26 Technical Challenges and Solutions34:29 The Challenge for Product Managers in AI34:55 Billy's Background in AI35:42 The Rapid Evolution of AI Technology37:14 Managing Information Overload37:58 Collaboration Between Product Managers and Engineers38:42 Trainline's Approach to Machine Learning39:36 Scaling Up: From 450 to 700,000 Pages40:21 Challenges in Data Retrieval and Processing45:55 Evaluating AI Assistants48:22 The Role of LLM as Judges50:19 User Context Simulation for Real-Time Evaluation01:06:56 Future Directions for Trainline's AI Assistant1 hr 9 min - Powering Government with Community Voices: How ZenCity Built an AI That Listens
Guests
- Noa Reikhav, Head of Product, Zencity
- Andrew Therriault, VP of Data Science, Zencity
- Shota Papiashvili, SVP of R&D, Zencity
In this episode
- How Zencity helps local governments reach, understand, and act on community voices
- Turning thousands of survey responses, social posts, 311 calls, and news items into usable insight
- Building a data model with multiple layers—raw data → elements → highlights → insights → briefs
- Why context is everything when building AI for civic use
- How the team designed their AI assistant using MCP servers to safely negotiate data access
- Balancing agentic flexibility with deterministic trust
- Evaluating accuracy when latency matters: how they think about evals, citations, and model-as-judge systems
- Using workflows like annual budgeting or crisis communication to deliver AI-generated briefs to the right people at the right time
- Why government workflows are the ultimate “jobs to be done” framework
Takeaways
- Data architecture defines what AI can do.
- Guardrails and transparency matter more than flashy outputs.
- Agentic systems become powerful when grounded in real, multi-tenant data.
- AI in the public sector can make democracy more responsive—if built responsibly.
Chapters:
00:00 Introduction to the Team00:16 What is ZenCity?01:26 AI in ZenCity's Platform06:00 Survey Methodologies and Use Cases09:01 Community Voices and Social Listening14:36 Workflows and AI Integration22:15 Annual Budget Planning Workflow32:44 Data Layers and Sentiment Analysis33:53 Post Interaction Surveys and Resident Engagement34:20 Data Enrichment and Sentiment Analysis35:14 Topic Modeling and Semantic Search36:50 AI Content Summarization and User-Driven AI Assistant38:53 Highlights, Insights, and the Gold Layer41:19 Challenges and Solutions in AI Data Processing46:47 AI Assistant and Guardrails01:05:27 Future Developments and Orchestration Layer01:06:44 Conclusion and Final Thoughts1 hr 8 min - Building AI Coworkers: How Neople Is Making Agents Work Where You Work
Guests:
- Seyna Diop, Chief Product Officer, Neople
- Job Nijenhuis, CTO & Co-founder, Neople
- Christos Constantinou, Lead Design Engineer, Neople
Chapters:
00:00 Meet the Team: Introducing Neople's Key Players01:16 Understanding Neople's Product: Digital Coworkers03:25 The Origin Story: How Neople Came to Be06:24 Customer Success: Real-World Applications of Neople12:36 Evolution of Neople: From Suggestions to Automations15:18 Integrations and Scaling: Making Neople Work Everywhere20:33 Ensuring Quality: Feedback and Evaluation Mechanisms37:13 Ensuring Quality in Customer Knowledge Bases37:45 Visibility and Transparency in Neople Responses39:42 Customer Feedback and Error Analysis42:57 Technical Details of Knowledge Retrieval49:15 Embedding Strategies and Document Types57:06 Automation and Actions in Customer Support01:06:37 Expanding Beyond Customer Support01:11:02 Conclusion and Future DirectionsLinks & References:
- Neople.io – Learn more about Neople’s AI coworkers
- The Joy Lab – Neople’s community and podcast about AI and work
1 hr 13 min - Building Alyx: How Arize AI Dogfooded Its Way to an Agentic Future
Guests:
- SallyAnn DeLucia, Director of Product, Arize
- Jack Zhou, Staff Engineer, Arize
In this episode, we cover:
- What tracing, observability, and evals really mean in GenAI applications
- How Arize used its own platform to build Alyx, its AI agent
- The role of customer success engineers in surfacing repeatable workflows
- Why early prototyping looked like messy notebooks and hacked-together local apps
- How dogfooding shaped Alyx’s evolution and built confidence for launch
- Why evals start messy, and how Arize layered evals across tool calls, sessions, and system-level decisions
- The importance of cross-functional, boundary-spanning teams in building AI products
- What’s next for Alyx: moving from “on rails” workflows to more autonomous, agentic planning loops
Resources & Links
- Arize AI — Sign up for a free account and try Alex
- Arize Blog — Lessons learned from building AI products
- Maven AI Evals Course — The course Teresa took to learn about evals (Get 35% off with Teresa’s affiliate link)
- Cursor — The AI-powered code editor used by the Arize engineering team
- DataDog — For understanding application traces
- OpenAI GPT Models — GPT-3.5, GPT-4, and newer models used in early and current versions of Alex
- Jupyter Notebooks — A tool for combining code, data, and notes, used in Arise’s prototyping
- Axial Coding Method by Hamel Husain — A framework for analyzing data and designing evals
Chapters:
00:00 Introduction to Sally Ann and Jack01:08 Overview of Arize.ai and Its Core Components01:44 Deep Dive into Tracing, Observability, and Evals03:56 Introduction to Alyx: Arize's AI Agent04:15 The Genesis and Evolution of Alyx08:51 Challenges and Solutions in Building Alyx24:33 Prototyping and Early Development of Alyx26:22 Exploring the Power of Coding Notebooks26:51 Early Experiments with Alyx27:59 Challenges with Real Data29:20 Internal Testing and Dogfooding31:55 The Importance of Evals35:16 Developing Custom Evals43:09 Future Plans for Alyx47:59 How to Get Started with Alyx50 min - Debugging AI Products: From Data Leakage to Evals with Hamel Husain
Guest: Hamel Husain
AI products and problems discussed:
- GitHub Copilot
- Forecasting AirBnB Guest Growth
- NurtureBossResources & Links
- Hamel’s blog on AI evals
- AI Evals for Engineers and PMs course on Maven (Get 35% off with this affiliate link)
Chapters:
00:00 Introduction to Hamel Hussein00:34 Challenges in AI Consulting02:00 Machine Learning Fundamentals04:47 Debugging Machine Learning Models05:00 Case Study: Airbnb's Guest Growth08:51 Understanding Machine Learning Models18:35 Introduction to Nurture Boss25:40 Building AI Products with Synthetic Data41:20 Connecting Machine Learning to Error Analysis42:28 Real-World Example: Text Message Errors44:15 Prioritizing and Documenting Errors45:59 Continuous Improvement and Iteration58:08 Using Synthetic Data for Evaluation01:08:42 Avoiding Overfitting in Evaluations01:19:28 Practical Tips for Error Analysis01:25:10 Final Thoughts and Resources1 hr 27 min - Inside eSpark’s AI Teacher Assistant: RAG, Evals, and Real Classroom Needs
Guests:
- Thom van der Doef, Principal Product Designer at eSpark
- Mary [last name], Director of Learning Design & Product Manager at eSpark
- Ray Lyons, VP of Product & Engineering at eSpark
Topics covered:
- The origin story of Teacher Assistant: connecting administrator mandates with teacher needs
- Why the team abandoned a chatbot interface in favor of a more structured workflow
- How retrieval augmented generation (RAG) and embeddings shaped the product architecture
- Lessons learned from debugging semantic search vs. keyword search
- Building evals with rubrics, Braintrust, and a human-in-the-loop approach
- What’s next for Teacher Assistant: more contextual recommendations using student data
Links & References:
- eSpark Learning
- Braintrust – evals and observability for LLM applications
- AI Evals Course by Hamel Husain and Shreya Shanker (Get 35% off with my affiliate link)
Chapters:
02:05 Overview of Epar's Adaptive Learning Program07:19 Challenges and Insights from COVID-1917:06 Developing the Teacher Assistant Feature24:55 User Experience and Interface Evolution34:29 Chat GPT-5's New Features35:16 Balancing Engagement and Efficiency35:40 Seasonal Business and Real Traffic36:29 Technical Decisions and RAG Implementation38:28 Challenges with Embeddings and Metadata41:24 Improving Recommendations and Data Enrichment55:18 Evaluating the Teaching Assistant01:05:51 Future Plans and User Feedback01:07:57 Conclusion and Final Thoughts1 hr 9 min
About Just Now Possible
From the publisher's feed
How AI products come to life—straight from the builders themselves. In each episode, we dive deep into how teams spotted a customer problem, experimented with AI, prototyped solutions, and shipped…
real features. We dig into everything from workflows and agents to RAG and evaluation strategies, and explore how their products keep evolving. If you’re building with AI, these are the stories for you.