Building Out Loud: Shipping vs. Feature Factory in an AI-Native World

Randy Silver and Faith Foster discuss Faith’s week after attending Product Con’s AI Leadership Conference, where she learned how regulated companies like BT are transforming legacy stacks for an AI-agent world and met the founder of Jack and Jill, an AI-native recruitment business that has automated its customer feedback-to-fix loop using multiple agents. 

They debate whether this kind of automation creates a sustainable product strategy or an “infinite feature factory,” and what observability and review are needed. 

Faith reflects that she overbuilt her product, making it far more complex, and recognises she has lost clarity on her ICP amid investor conversations and the spectrum of AI maturity across businesses. 

With customers waiting, she commits to simplifying and shipping a beta, using real usage—and eventually asking for money—as the forcing function for learning.

00:00 Welcome and weekly check-in
00:30 ProductCon insights on AI transformation
02:07 Automated feedback loops and feature factories
03:52 Discovery focus and corporate expectations
04:28 Overbuilding and losing the ICP
06:20 Alpha learnings and investor pressure
08:48 Back to hypotheses and real users
09:38 Solo founder tension and accountability
12:08 Poker lesson ask for money
13:10 This week ship the beta
13:57 Wrap-up and next week challenge

Building Out Loud: Rethinking PRDs, Roadmaps, and Product Teams in an AI-Native World

Randy Silver checks in with Faith Forster on her progress building Discovery after speaking with over 70 product leaders and using feedback to update the product, shifting it toward a “strategic advisor” and embedding agents to help shape and apply product strategy across decision-making. 

They discuss how AI changes product development fundamentals—what roadmaps, PRDs, epics, and tasks mean when building is faster and work may be defined for agents rather than engineers—highlighting a “product experience document” that emphasises behavior, tone, and guardrails. Faith explains Discovery’s approach: defining problems, generating and comparing solution ideas, then producing different scoping/design outputs depending on audience (engineer, AI agent, stakeholder), with customisable templates. 

They also cover collaboration, bias, diversity of viewpoints, the limits of artificial personas versus real users, prototyping trade-offs, and challenges for legacy and regulated stacks. 

Faith’s next priority is getting the beta ready around ProductCon.

00:00 Weekly Build Update
01:21 Future of Product Work
03:50 Teams to Domains Shift
04:01 Artifacts Not Outcomes
06:30 Designing for Agents
09:14 Collaboration and Bias
12:35 Build Fast Risk Smart
15:46 Prototypes and Legacy
17:51 Beta Prep and Wrap

Building Out Loud: Beta Prep, Investor Decks & Fixing AI Agents with Langfuse

In this episode of Building Out Loud, Randy Silver and Faith Forster recap a big week: Faith meets Anton, founder of Lovable, and reflects on where Lovable excels (websites and decks) but still falls short for production B2B apps. 
Buoyed by strong alpha sign-ups, she begins preparing to raise funding, sharing an investor deck and getting early feedback from investors who prefer post-revenue companies. 
On the product side, she finally improves her agent reliability by combining guidance from Claude with Replit and by adding Langfuse tracing and evals for clearer debugging than basic logs. 
Faith then previews rapid beta progress, including a revamped “Competitor Intelligence” area with dynamic profiles, strategic analysis, comparisons, and assignable product recommendations, and plans to have the beta ready for upcoming team onboarding calls.

00:00 Welcome Back Update
00:24 Meeting Lovable Founder
01:09 Pitch Deck And Fundraising
02:47 Agents Finally Working
04:06 How Technical To Be
07:19 Replit Logs Tour
08:11 Langfuse Tracing And Evals
12:40 Beta Build Sneak Peek
13:17 Competitor Intelligence Features
18:32 Roadmap And Wrap Up

Building Out Loud: Debugging AI Agents, Evals, and When to Move Beyond Prototyping Tools (with Ed Biden of Hustle Badger)

Randy Silver and Faith Forster are joined by Ed Biden, co-founder of Hustle Badger, to follow up on Faith’s progress building an AI-powered discovery product. 
Faith shares insights from roughly 20 interviews and survey feedback, noting strong resonance with the product’s commercial outcomes value proposition but a clear need to improve trust in the underlying data by getting her “army of agents” working reliably before polishing features. 
They discuss the temptation to immediately implement feedback versus waiting to prioritise patterns, and Faith details practical debugging issues including missing notifications (e.g., running out of Perplexity credits), JSON parsing problems, and OpenAI producing no output. Ed explains four categories of AI building tools (LLMs, workflow builders, prototyping tools, IDEs) plus an emerging fifth category, then introduces evals and observability via traces, clustering failure modes, and automated tests. 
Faith plans to try Langfuse, improve dashboards, rethink agent orchestration with planning agents, and continue toward beta users while balancing automation with team control.

Check out Hustle Badgers Evals course: https://youtu.be/TA9LJJddlNE
Or watch their introductory video on Evals: https://youtu.be/7OcrV7VSvW4

00:00 Welcome and Guest Intro
00:49 Discovery Progress Update
01:33 Resisting Quick Fixes
02:57 Agent Debugging Woes
03:43 Moving Beyond Replit
05:03 AI Tooling Landscape
08:30 Swarms and Subagents
09:16 Evals and Observability
15:05 Building Better Evals
18:21 Faith Next Steps and Roadmap
21:29 Replit vs IDE Comfort
23:05 Wrap Up and Teaser

Building Out Loud: 70 Alpha Signups, Early Validation & Getting to a Production-Ready Stack

Randy Silver and Faith Foster check in on Faith’s weekly progress building her AI product discovery tool. 

Faith has grown her alpha group to 70 people, with about half of the original 60 logging in and nine completing the feedback form; 55% rated it extremely valuable or valuable, and about 80% saw value overall. Feedback highlights that adoption is a multi-stakeholder, team-wide decision and that onboarding and implementation need to be seamless, including importing existing product documentation and expanding the AI assistant to create roadmap items. 

Faith updated messaging and a new website to position it as a strategic CPO purchase focused on CEO/board decision-making and commercial alignment, prioritising UK/EU B2B SaaS (especially PE-backed). Polarised reactions informed a two-tier pricing model (tactical feedback/roadmapping vs outcome and goal alignment). 

Next she’ll chase testers, prepare four beta users, improve agent reliability, data security, and integrations, and move to a more robust stack with evals tooling; a future episode will feature Ed Biden on evals and production readiness.

00:00 Weekly Check In
01:09 Early User Signals
02:20 Onboarding And AI Assistant
03:06 Messaging And Jobs To Be Done
04:30 Ideal Customer Profile
05:45 Maturity And Pricing Tiers
07:32 Next Week Execution Plan
08:20 Stack Upgrade And Evals
09:05 Confidence And Hiring Help
11:19 Next Episode Preview

Building Out Loud: Under the Hood of AI Agents (Logging, LLM Routing & Competitor Tracking)

Randy and Faith catch up on Faith’s startup progress after sending a trial to 60 people and getting useful feedback on product, selling, implementation, and how it fits into customers’ tool stacks. 

This episode goes under the hood on why shipping took longer: building reliable AI agents was frustrating and required re-architecting the platform. Faith demos a competitor-monitoring workflow that evolved from one failing “competitor agent” into 8–9 narrow agents covering sources, product summary, segments, news/updates, pricing, features, integrations, and review summaries. 

They discuss the need for rigorous logging, avoiding hard-coded behavior, and treating AI like a “lazy developer” that must be audited. Faith added editable prompts, scheduled/manual refresh, cost tracking, and an LLM router with fallback and bring-your-own-keys support, reducing Gemini errors from 48% to 19%, and plans a new website and product direction updates next week.

00:00 Welcome Back Setup

00:20 Trial Update Tease

00:56 Agent Dev Frustrations

01:48 Competitor Agent Breakdown

04:14 Pricing And Reviews Wins

06:32 Hardcoding And Trust

07:46 Logging Error Rates

09:58 Prompts Triggers LLM Switch

13:01 Time Estimates Rabbit Holes

14:18 Next Week Plans Wrap

Building Out Loud: Initial MVP Feedback & wrestling with Agents 

Randy Silver and Faith Forster catch up after Faith reaches a key milestone: a working MVP that covers the full workflow. 

After walking a few “friendly” product leaders through the tool, Faith uses early feedback to sharpen the purpose of each element. She shares strong early demand signals and early indications of willingness to pay.

Faith explains recent work on AI agents and why it has been harder than expected. Based on user conversations, Faith talks about some of the updates she has already made to deliver even more value. 

00:00 MVP Update: Full Workflow Working & Early Friendly Feedback
00:50 Sharpening Features: Rethinking the Roadmap Around Impact
01:38 Demand Signal: 53 Product Leaders Raise Their Hands
02:19 Will They Use It and Pay? Early Buyer Validation
04:12 Building with AI Agents: The Pain of Training Web Search & Pricing Scrapes
05:37 Refactoring the Agent Stack: Router + Choosing Perplexity/OpenAI/Gemini
06:36 Cadence & Cost: Scheduling Agents and Avoiding Market Noise
08:01 Competitor Intelligence: Threat Levels, Refreshes, and Instant Battle Cards
09:06 Assistant vs Autonomous Agents: What’s Hard, What Learns Over Time
09:39 Evals, Corrections, and Learning Loops in the Product
11:26 Next Week: Sending 60 Invites, Collecting Beta Feedback, and Messaging Prep
12:07 Wrap-Up and What’s Coming Next

Building Out Loud: Faith’s MVP Is Live — Debugging, Gemini Integration, and What Comes Next

In this episode of Building Out Loud, Randy Silver checks in with Faith after a gap in updates and she reveals big news: her MVP is now live in production. 

Faith shares that launching triggered a major debugging and security push, including ensuring a secure multi-tenant setup and correct data handling.  

Faith walks through the MVP. She demonstrates the setup for product teams, business goals, and team goals, enabling planned value realization tracking. 
At the team level, the app collects competitor and own-product feedback from review and comparison sites, groups insights by themes or shows raw feedback with source links, and supports creating opportunities manually or automatically. The roadmap emphasizes value realization through planned start dates and time investment, will sync with Jira in the future, and can generate PRDs or user stories from collected context. 

Randy challenges whether this is truly an MVP given how much is built and the fact that no users have tested it yet. Faith explains how AI drastically shortened build time, but agrees the next step is validating usability and fit. She plans to reach out to product leaders and senior product managers for hands-on feedback, focusing on whether the goal setup matches varied organisational realities and whether the core workflow—from feedback to opportunities to solutions to roadmap—fits real operating and governance cycles. 

After another revision round, she hopes to onboard teams for a beta. She expects feedback in a couple of weeks, will continue tidying up, start building integration capabilities, and use early feedback to revise the website, polish and publish the podcast episodes, and build a drumbeat about how AI is changing product decision-making.

00:00 Live MVP Launch: What’s Working (and What Broke in Production)
00:42 The Confidence Turning Point: Replit Support, Gemini Integration, and Reliability
02:03 MVP Tour: Dashboard, Competitors, and Roadmap Views
03:22 Goals & Metrics Setup: Connecting Team Work to Business Outcomes
04:22 Core Workflow Demo: Feedback Agents → Opportunities → Solutions
05:59 Roadmap Execution: Value Realization, Jira Sync, and AI-Generated PRDs
06:39 Is This Really an MVP? How AI Changes the Definition of “Minimum”
07:53 Testing Plan: Hypotheses, Open Questions, and What to Validate
10:56 What Success Looks Like: From Solo Testing to Team Alpha
12:10 Next Steps & Wrap-Up: Integrations, Website Messaging, and Next Check-In

Building Out Loud: Naming and Branding with Chad Butz

In this episode of 'Building Out Loud,' host Randy Silver and guest Chad discuss the journey of Faith's new startup, focusing on the challenges and processes involved in naming and branding the product 'Discoveree.' 

They explore the importance of a strong brand, product messaging, and the difficulties of naming a new product. The conversation delves into the iterative process of testing and refining the name and message, the role of AI in product development, and the overall strategy for launching a new product. Chad and Faith share insights from their previous experiences and emphasize the significance of early customer feedback and confidence-building for founders.

00:00 Introduction and Special Guest
00:41 The Importance of Branding
01:19 Naming the Product
05:00 Crafting the Messaging
07:55 Building the Website
10:03 Testing and Feedback
12:18 Comparing Past and Present Startups
13:30 Conclusion and Next Episode Teaser

Building Out Loud: Faith's Path to Discoveree - Naming, Prototyping, and Feedback

In this episode of 'Building Out Loud,' hosts Randy and Faith discuss her latest progress on the newly named app, Discoveree. Faith explains her decision to move development from Lovable to Replit for better functionality and reliability. She highlights Discoveree's new features, including onboarding flows, AI-generated opportunities, and sentiment scoring. Faith also shares her learnings from feedback sessions with product leaders and explores the delicate balance between adding features and validating the MVP. Tune in to hear about the next steps and challenges in bringing this AI-driven product management tool to life.

00:00 Introduction and Recap
00:32 Revealing the App Name
01:11 Choosing the Right Development Platform
02:09 Building the App: Features and Challenges
02:58 AI-Generated Opportunities and Feedback
04:01 Sentiment Scoring and API Integration
05:46 Validation and Future Plans
09:18 Conclusion and Next Steps