The Vernon Richard Show

Six Hard Lessons From Building With AI Agents


Listen Later

In this episode of the Vernon Richard show, the hosts discuss their experiences with AI tools and agents, focusing on the challenges and lessons learned from using these technologies in coding and software engineering. They explore best practices for utilizing AI effectively, the importance of context in interactions with AI, and the future of AI agents in the workplace. The conversation highlights the balance between leveraging AI for efficiency while maintaining control and understanding of the underlying processes.

Links to stuff we mentioned during the pod:

  • 09:16 - The LinkedIn post talking about Replit messing with someone's production code 😳
    • And the link to the thread of person who went through it
    • The tool in question, Replit
  • 13:01 - Rich's LinkedIn post with his tips
  • 14:21 - GitHub Copilot
  • 18:09 - VS Code
  • 29:01 - Folks at different ends of the "AI Enthusiasm Spectrum"
    • On the enthusiastic end
      • Jason Arbon is on the positive side and is always creating something interesting like...
        • testers.ai
    • On the unenthusiastic end
      • Keith Klain has created a reading list to help get us up to speed...
        • Keith's reading AI reading list
        • You can see his full resources list here
      • Maaike Brinkhof has a bunch of thought-provoking posts on the topic...
        • ...like this one
        • and this one
  • 34:44 - Want to know what "conflabulation" means? Listen to Martin explain it on the Ghost in th code podcast (that's not a typo!)
  • 37:24 - What is Context Engineering? Perplexity has answers!
  • 46:38 - The legendary Lt. Geordi La Forge from Star Trek: The Next Generation.
  • 51:48 - After recording, the very cool Paul Coles published his article The Subtle Art of Herding Cats: Why AI Agents Ignore Your Rules (Part 1 of 4, explaining the topic of Context Engineering. It’s brilliant!
  • 59:04 - The promises of technology over the years...
  • 60:50 - The always insightful Meredith Whittaker of Signal fame, where is the president and services on its board of directors, explains the privacy and security concerns with agentic technology.


    • Watch the clip, then go back and watch the whole thing!

00:00 - Intro
01:17 - Welcome
01:30 - TANGENT BEGINS... All kinds of egregious waffling follows. Skip to the actual content at 08:34
01:31 - Rich VS Tree Stump
01:57 - What on earth did Rich need the pulley for?
02:26 - Vern's nerdy confession and pulley confusion
02:52 - Does Rich live next door to Tony Stark?!
03:22 - What to do when you need a steel RSJ
03:35 - We admit defeat. 03:36 - Welcome to Rich's Garden Adventures Podcast!
07:25 - What has Vern been up to?
08:34 - We attempt to segue into the episode at last!
08:35 - TANGENT ENDS...
08:51 - Rich’s POC: using agents to help build AI tools
09:45 - The Replit disaster: vibe coding meets deleted production data 11:12 - Sociopathic assistants and the case for AI gaslighting 11:55 - Vernon wants his team experimenting with AI tools
12:50 - Rich explains the context for his latest AI adventures
13:18 - Rich’s bench project and “putting the engineering hat on” 
15:22 - Setting up the stack and staying in control 
16:53 - A familiar story: things were going fine until they weren’t 
17:00 - Ask vs Edit vs Agent mode in Copilot explained 
19:06 - The innocent linting error that spiralled out of control 
21:16 - Stuck in a loop: “I didn’t know what it was doing, but I let it keep going” 
22:11 - The fateful click: “I’m going to reset the DB” 
23:10 - The aftermath: no data, no damage
 but very nearly 
23:33 - Security wake-up call: agents are acting as you 
24:39 - You can’t fix what you don’t know it broke 
25:52 - Can you interrupt an agent mid-task? 
27:14 - When agents get “are you sure?” moments 
28:15 - Tea breaks as a dev strategy: outsourcing work to agents 
29:24 - Jason Aborn vs Keith & Maaike: where Rich sits on the AI enthusiasm spectrum 
30:41 - Tip1. The first of Rich’s 6 agent tips: commit after every interaction
32:12 - Why trusting the “keep all” button is risky 
34:01 - Writing your own commits vs letting the agent do it 
35:26 - When agents lose the plot: reset instead of fixing 
36:55 - “You’re insane now, GPT. I’m giving you a break.” 
37:54 - Tip 2: Make the task as small as possible 
39:59 - The middle ground between 'ask' and full agent delegation 
41:12 - Tip 3: Ask the agent to break the task down for you 
43:36 - The order matters: why you shouldn’t start with the form UI 
44:33 - Vernon compares it to shell command pipelines 
45:09 - It can now open browsers and run Playwright tests (!) 
46:23 - Star Trek and the rise of the engineer-agent hybrid 
47:57 - Tips 4–6: Test often, review the code, use other models 
49:39 - Pattern drift and the importance of prompt templates 
50:51 - Vernon’s nemesis: m dashes, emojis, and being ignored by GPT 
51:48 - Context engineering vs prompt engineering 
52:43 - When codebases get too big for agents to cope 
53:40 - Why agents sometimes act dumber than your IDE 
54:32 - The danger of outsourcing good practices to AI 
54:48 - Spoilers: Rich’s upcoming keynote at TestIt 
55:01 - Agents don’t ask why — they just keep going 
56:42 - Goals vs loops: when failure isn’t part of the plan 
58:32 - The question of efficiency: is training agents worth it? 
59:47 - Rich’s take: we’ll buy agents like we buy SaaS 
61:08...

...more
View all episodesView all episodes
Download on the App Store

The Vernon Richard ShowBy Vernon Richards and Richard Bradshaw


More shows like The Vernon Richard Show

View all
Test Match Special by BBC Radio 5 Live

Test Match Special

240 Listeners

More or Less: Behind the Stats by BBC Radio 4

More or Less: Behind the Stats

896 Listeners

The Martin Lewis Podcast by BBC Radio 5 Live

The Martin Lewis Podcast

71 Listeners

No Such Thing As A Fish by No Such Thing As A Fish

No Such Thing As A Fish

4,834 Listeners

Testing Peers by Testing Peers

Testing Peers

0 Listeners

The Rest Is Politics by Goalhanger

The Rest Is Politics

3,094 Listeners

The Rest Is Politics: US by Goalhanger

The Rest Is Politics: US

2,209 Listeners