Behind the Craft

Today, I want to share a new episode with Aman Khan.The best way to learn about AI evaluations is to watch 2 PMs build them live from scratch. In our new episode, Aman and I walk through creating evals for an AI customer support agent — from labeling a golden dataset to aligning LLM judges. This is the complete beginners AI eval course you've been waiting for.Aman and I talked about:

(00:00) What are AI evals and how to get good at them

(02:52) The 4 types of AI evaluations everyone should know

(06:08) Live demo: Building evals for a customer support agent

(10:29) Using Anthropic's console to generate great prompts

(15:13) Creating the evaluation criteria

(17:40) Adding human labels to the golden dataset

(31:05) Scaling evals with LLM-judge prompts

(38:21) How to align LLM judges with human judgmentGet the takeaways: https://creatoreconomy.so/p/complete-beginner-course-on-ai-evaluations-aman-khanWhere to find Aman:

X: https://www.linkedin.com/in/amanberkeley/

Website: https://arize.com/📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Lee Robinson.Lee is the VP of Developer Experience at Cursor and a legend in his field.

While most people are still vibe coding, Lee showed me how to build apps properly using planning, test-driven development, AI agent workflows, and more. This is a must-watch tutorial for beginners who want to learn how to build with AI the right way.Lee and I talked about:

(00:00) How the way we build software is fundamentally changing

(03:49) Live demo: Building a music tracking app with Cursor

(5:17) The critical setup phase most people skip (tests, git, linting)

(16:03) Foreground vs background agents and when to use each

(27:34) How to create cursor rules that compound your workflows

(37:31) Why experienced engineers shouldn't fear AI tools

(42:19) Essential advice for PMs who want to learn technical skills

(44:48) How Cursor builds products without any PM

(47:18) The future of generalist engineers in an AI world

Get the takeaways: https://creatoreconomy.so/p/cursor-tutorial-for-beginners-from-cursor-head-of-ai-education-lee-robinson

Where to find Lee:

X: https://x.com/leerob

Website: https://leerob.com/

📌 Subscribe to this channel – more tutorials coming soon!

Today, I want to share a new episode with Ben Erez.Ben is a former Meta PM who has helped 100s of people ace their PM interviews and land their dream jobs in tech. I got him to show me exactly how to ace product sense interviews and build an AI-copilot that his students say feels like having a "calibrated interviewer that's available 24/7."Ben and I talked about:(00:00) The current state of the PM job market

(02:07) The exact structure of the product sense interview

(08:58) How to build your AI interview co-pilot step-by-step

(12:36) Live demo: AI co-pilot walking through a Meta interview question

(13:33) Why Ben designed the AI to be the candidate, not the interviewer

(25:23) How to prioritize the right user segments and problems to solve

(32:13) The biggest mistakes candidates make when challenged by interviewersThis episode covers how to build a simple AI co-pilot from scratch, but you should also check out Ben’s advanced interview AI co-pilot: https://www.benerez.com/copilotGet the takeaways: https://creatoreconomy.so/p/how-to-build-an-ai-co-pilot-to-ace-your-pm-interviews-ben-erezWhere to find Ben:

LinkedIn: https://www.linkedin.com/in/benerez/

Website: www.benarez.com📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Jeremy Epling.

Jeremy is the CPO of Vanta and former VP of Product at Github. He showed me the exact template that he used to get his entire team to prototype with AI, his favorite AI workflows for PM work, and why now might be the best time to be an IC PM. We also did a deep dive on how Vanta ran AI evaluations for their new AI agent.

Jeremy and I talked about:(00:00) Why prototypes beat PRDs

(01:43) The exact template that got his entire PM team prototyping

(05:14) From prototype to product: live demo of Vanta’s AI agent

(08:56) How to run prototype-first product reviews

(12:47) Top AI workflows that save Jeremy hours each week

(19:08) Is now the best time to be an IC PM?

(25:03) Advice for big tech PMs facing industry changes

(30:35) Separating the hype from craft for AI products

(36:16) Deep dive on AI evaluations for AI agents

Where to find Jeremy:

LinkedIn: https://www.linkedin.com/in/jeremy-epling-j40/

Website: https://vanta.com Get the takeaways: https://creatoreconomy.so/p/how-to-get-your-team-to-start-prototyping-with-ai-jeremy-vanta📌 Subscribe to this channel – more interviews coming soon!

Meng is the founder of Aura, an AI prototyping tool for designers. He has spent 100s of hours using AI to create beautiful designs that don’t look like generic AI slop. Watch as he shares his exact process in this episode — from prompting to layouts to animations, and more without having to code.Timestamps:

(00:00) Why most AI websites look generic and how to fix it

(01:19) Comparing prompts across V0, Lovable, and Aura

(06:42) Using references and templates instead of blank prompts

(08:54) Building a great website prompt from scratch

(16:51) Why AI should augment, not replace your creativity

(22:43) Adding testimonial sections and scrolling animations

(29:40) How to add Spline 3D backgrounds step-by-step

(36:12) The 90/10 rule: When to use AI vs manual tweaksGet the takeaways: https://creatoreconomy.so/p/use-ai-to-create-beautiful-designs-meng-to

Where to find Meng:

X: https://x.com/MengTo

Website: https://aura.build/📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Kieran Klaassen.Kieran is the founder of Cora, a beautiful AI email assistant. While we’ve all been vibe coding, Kieran has gone to the next level to manage multiple AI agents to code for him at the same time. I got him to show me exactly how he does this using Claude Code, voice, slash commands, and more. You don’t want to miss this episode 🙂Timestamps:(00:00) Why agentic coding beats vibe coding(03:21) "It's like I'm a manager and I have an AI agent team"(06:29) Why Claude Code is winning over Cursor for power users(12:15) Live demo: Building a new feature with 3 AI agents in parallel(22:32) Custom slash commands to eliminate repetitive work(36:13) AI code reviews using multiple perspectives and bots(54:55) Inside Cora: Your AI Chief of Staff for email(1:00:11) Advice for non-engineers intimidated by agentic codingGet the takeaways: Where to find Kieran:X: https://x.com/kieranklaassenWebsite: https://cora.computer/📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Morgan Brown.Morgan is the VP of Product for AI at Dropbox. He gave me an inside look at his top AI workflows. We also talked about building Dash, Dropbox’s new AI agent for work, and how AI will reshape PM.Timestamps:

(00:00) How Morgan uses AI as a companion for everything he does

(01:26) Morgan's 3 best AI workflows to save time at work

(06:07) Inside Dash: Dropbox's AI agent to find anything at work

(14:35) The evaluation process behind Dash's search results

(18:49) How to ship fast while motion doesn't equal progress

(22:09) How to balance internal alignment with customer obsession

(27:27) What Morgan learned from Drew (Dropbox), Tobi (Shopify), and Zuck

(35:39) The 3 types of PMs Morgan looks for in the AI era

(38:02) What product management will look like in one yearWhere to find Morgan:

LinkedIn: https://www.linkedin.com/in/morganb/

Website: https://dash.dropbox.com/Get the takeaways: https://creatoreconomy.so/p/dropbox-vp-best-ai-workflows-to-take-back-time-morgan-brown 📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with James Evans.

James is the Head of AI for Amplitude, the leading product analytics platform. We had a great chat about why nobody has built a good AI analyst yet and why he’s betting on AI agents that can monitor your product 24/7. James also shared some real talk about whether AI will displace PM and analyst jobs.

Timestamps:

(00:00) Why nobody has built a good AI data analyst yet

(03:32) The biggest problem with product analytics today

(06:13) Live demo: AI agents that monitor your website 24/7

(21:48) The hardest parts of building AI analytics products

(25:21) How to evaluate AI agents that run experiments

(32:53) AI product pricing strategies that actually work

(35:42) Should data scientists worry about their jobs?

(38:04) Non-obvious advice for building AI products

Get the takeaways: https://creatoreconomy.so/p/is-this-the-first-ai-analyst-that-actually-works-amplitude-james-evans

Where to find James:

LinkedIn: https://www.linkedin.com/in/james-evans-7086b3126/

Website: https://amplitude.com/ai

📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Cameron Adams.Cameron leads product for Canva where people are creating 1B+ designs every month. We talked about how AI will transform design and why he believes in prototypes over PRDs and coaches over managers. I also loved hearing Cam’s stories about Canva’s rubber ducky easter egg and what a bike accident taught him about persistence.

This episode is brought to you by Vanta — Join 9K+ companies like Atlassian who use Vanta to manage risk and prove security in real-time. Get $1000 off at https://vanta.com/peter

Timestamps:

(00:00) Will AI democratize design?

(06:32) How Canva runs AI evaluations for its product

(10:17) Moving beyond chat boxes to visual AI interfaces

(22:23) Why Canva skips PRDs and goes straight to prototypes

(29:08) The rubber duck Easter egg that became a cultural icon

(33:13) Keeping craft and quality as a priority as the team grows

(35:32) Why Canva gives everyone executive coaching, not just managers

(39:23) How AI is blurring boundaries between PM and design

(43:26) The bike accident story that almost derailed Canva's launch

Get the takeaways: https://creatoreconomy.so/p/canva-co-founder-ai-playbook-most-popular-design-tool-cameron-adams

Where to find Cameron:

LinkedIn: https://www.linkedin.com/in/themaninblue/

Website: https://www.canva.com/ai-assistant/

📌 Subscribe to this channel – more interviews coming soon!

Today, I want to share a new episode with Nate Gonzalez.Nate leads ChatGPT for Work which is now used by 92% of Fortune 500 companies. In our chat, he reveals how OpenAI runs with less than 30 PMs, what they look for in new hires, how he personally uses ChatGPT to save time at work, and more.

Timestamps:(00:00) 92% of Fortune 500 companies already use ChatGPT

(02:06) OpenAI's latest features for ChatGPT at work

(14:52) Why OpenAI has less than 30 PMs for 5,000 employees

(15:58) What traits OpenAI looks for when hiring PMs

(18:31) The 10-minute AI hack that changed how Nate works

(25:56) The most surprising thing about working at OpenAI

(29:48) The biggest barriers to AI adoption and how to overcome them

(38:36) Using ChatGPT roleplay to prep for important meetings

(41:04) ChatGPT's future: From assistant to trusted coworker

(43:06) The specific skill that will keep your job safe in the AI eraGet the takeaways: https://creatoreconomy.so/p/how-openais-head-of-business-products-uses-chatgpt-at-work-nate-gonzalezWhere to find Nate:

LinkedIn: https://www.linkedin.com/in/nate-gonzalez/

Website: https://openai.com/chatgpt/enterprise/📌 Subscribe to this channel – more interviews coming soon!