Product Growth Podcast

How to Build Multi-Agent AI Systems That Actually Work in Production | Tyler Fisk


Listen Later

Tyler Fisk built a $1.6 million AI education business in one year.

Zero PhDs. Zero Silicon Valley pedigree. Just a systematic approach to building AI agents that actually work in production.

While everyone’s vibe coding in ChatGPT, Tyler’s teaching thousands of students to build multi-agent systems for real businesses. Hundreds of production deployments. Actual revenue.

Today he’s doing a live build: Taking Apple customer service from idea to working multi-agent system in under 90 minutes.

No theory. Pure execution.

----

Brought to you by:

* 1. Maven: Get $135 off Tyler’s course with my code AAKASHxMAVEN

* Vanta: Get $1,000 off AI security & compliance at vanta.com/acos

* Testkube:

* Kameleoon: Leading AI experimentation platform

* The AI Evals Course for PMs: Get $1155 off with code ‘ag-evals’

----

Key takeaways:

1. Stop Vibe Coding: Most teams write one prompt, test twice, ship to production, and hope for the best. Tyler's rule: "We would never put it into production without a human-in-the-loop checkpoint. That's irresponsible." Start with 100% human review, gradually move to 60-70% autonomy.

2. Use Meta-Prompting to Build Agents: Tyler built Gigawatt—an agent with 72,000 characters of system instructions that builds other agents. It researches the domain, writes V1 instructions, evaluates itself (scores out of 100), identifies gaps, and rewrites to V2. Goes from 77% to 86%+ quality.

3. Build Multi-Agent Architectures: Don't build one agent that does everything. Separate concerns like you'd separate teams. For Apple: Core (expert agent, temp=0, finds facts) + Echo (email agent, temp=0.7, writes responses). Each optimized for its specific role.

4. System Instructions Need 7K-9K Tokens: Structure includes Role (job description), Context (business details), Instructions (step-by-step process), Criteria (guardrails), Examples (meta reasoning). Most people write 200 tokens. Tyler writes 7,000-9,000. That's the foundation.

5. Temperature Is Your Secret Weapon: Tyler's Toy Story analogy: Imagine an icy peak in a claw machine. Temp=0 (frozen): claw picks from top only—deterministic, precise. Temp=1 (melted): claw grabs anywhere—creative, varied. Match temperature to agent's job.

6. Information Hierarchy Prevents Hallucinations: Priority order: RAG database first (scraped company docs), System instructions second (built-in expertise), Web search third (with chain-of-verification). When agents search without verification, they hallucinate.

7. Build Complete Workflows: Tyler's 9-step production workflow with 5+ agents: Email arrives → Sentiment analysis (Cinnamon) → Expert research (Core) → Email writing (Echo) → QA loop → Human checkpoint (Slack) → Generative filter → Send → Log to memory.

8. Observational Evals Come First: Test 20+ different scenarios manually. Include edge cases and adversarial inputs. Document every failure. Save golden examples. Only after building confidence do you add systematic evals in production.

9. Calculate ROI as Labor Cost Reduction: Traditional cost: $460/day (expert time + customer service rep + manager review) = $138K/year. AI cost: $153/day (platform fees + API credits + human review) = $45.9K/year. Savings: $92K annual (67% reduction).

10. Emotion Prompting Actually Works: Tyler ends every prompt with "Go get 'em slugger." Based on research: positive reinforcement improves LLM outputs by ~15%. The same psychology that works on humans works on LLMs. "Be nice to your AI. They're gonna have robot bodies soon."

----

Related Content

Podcasts:

* Warp CEO on Profitable AI Agents

* Elizabeth Laraki on AI Product Design

* Claude Code Tutorial

Newsletters:

* AI Agents: The Ultimate Guide for PMs

* How to Build AI Products Right

* Ultimate Guide to AI Prototyping Tools

----

P.S. More than 85% of you aren’t subscribed yet. If you can subscribe on YouTube, follow on Apple & Spotify, my commitment to you is that we’ll continue making this content better.

----

If you want to advertise, email productgrowthppp at gmail.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe
...more
View all episodesView all episodes
Download on the App Store

Product Growth PodcastBy Aakash Gupta

  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

27 ratings


More shows like Product Growth Podcast

View all
This Week in Startups by Jason Calacanis

This Week in Startups

1,290 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

533 Listeners

The Official SaaStr Podcast: SaaS | Founders | Investors by SaaStr

The Official SaaStr Podcast: SaaS | Founders | Investors

172 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

1,085 Listeners

Masters of Scale by WaitWhat

Masters of Scale

3,986 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

234 Listeners

Product Thinking by Melissa Perri

Product Thinking

147 Listeners

The Startup Ideas Podcast by Greg Isenberg

The Startup Ideas Podcast

203 Listeners

Lenny's Podcast: Product | Career | Growth by Lenny Rachitsky

Lenny's Podcast: Product | Career | Growth

1,362 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

131 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

94 Listeners

AI and I by Dan Shipper

AI and I

38 Listeners

AI + a16z by a16z

AI + a16z

33 Listeners

Uncapped with Jack Altman by Alt Capital

Uncapped with Jack Altman

39 Listeners

How I AI by Claire Vo

How I AI

146 Listeners