Sign up to save your podcasts
Or
Share The AI Skill That Will Define Your PM Career in 2025 | Aman Khan (Arize)
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
The AI Skill That Will Define Your PM Career in 2025 | Aman Khan (Arize)
My guest today is Aman Khan.
Aman is the Director of Product at Arize AI. Last year, we both heard the Chief Product Officers of OpenAI and Anthropic share that AI evaluations will be the most important skill for PMs in 2025. Aman gave me a crash course on how to build this critical skill in our interview.
Timestamps:
(00:00) Evals force you to get into your user's shoes
(03:42) 5 skills to build right now to become a great AI PM
(07:43) How curiosity leads to better AI products
(10:20) You can build AI without the job title
(17:00) A deep dive into why AI evals are so important
(24:53) Example of running evals for an AI customer support agent
(31:32) When humans need to be in the loop for evals
(35:46) How to get better at writing evals right now
(39:39) My personal prompt and eval for transcripts
(44:23) Catching the AI wave is alot like surfing
Get the takeaways: https://creatoreconomy.so/p/the-ai-skill-that-will-define-your-pm-career-aman-khan
Where to find Aman:
LinkedIn: https://www.linkedin.com/in/amanberkeley/
Arize: https://arize.com/
📌 Subscribe to this channel – more interviews coming soon!
View all episodes
By Peter Yang
5
55 ratings
My guest today is Aman Khan.
Aman is the Director of Product at Arize AI. Last year, we both heard the Chief Product Officers of OpenAI and Anthropic share that AI evaluations will be the most important skill for PMs in 2025. Aman gave me a crash course on how to build this critical skill in our interview.
Timestamps:
(00:00) Evals force you to get into your user's shoes
(03:42) 5 skills to build right now to become a great AI PM
(07:43) How curiosity leads to better AI products
(10:20) You can build AI without the job title
(17:00) A deep dive into why AI evals are so important
(24:53) Example of running evals for an AI customer support agent
(31:32) When humans need to be in the loop for evals
(35:46) How to get better at writing evals right now
(39:39) My personal prompt and eval for transcripts
(44:23) Catching the AI wave is alot like surfing
Get the takeaways: https://creatoreconomy.so/p/the-ai-skill-that-will-define-your-pm-career-aman-khan
Where to find Aman:
LinkedIn: https://www.linkedin.com/in/amanberkeley/
Arize: https://arize.com/
📌 Subscribe to this channel – more interviews coming soon!
More shows like Behind the Craft
View all
This Week in Startups
1,292 Listeners
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
536 Listeners
The Official SaaStr Podcast: SaaS | Founders | Investors
172 Listeners
a16z Podcast
1,091 Listeners
Founders
2,112 Listeners
Y Combinator Startup Podcast
236 Listeners
The Startup Ideas Podcast
205 Listeners
No Priors: Artificial Intelligence | Technology | Startups
131 Listeners
AI and I
37 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
515 Listeners
AI + a16z
34 Listeners
Lightcone Podcast
22 Listeners
Training Data
39 Listeners
Uncapped with Jack Altman
39 Listeners
Cheeky Pint
43 Listeners