Behind the Craft

The AI Skill That Will Define Your PM Career in 2025 | Aman Khan (Arize)


Listen Later

My guest today is Aman Khan.

Aman is the Director of Product at Arize AI. Last year, we both heard the Chief Product Officers of OpenAI and Anthropic share that AI evaluations will be the most important skill for PMs in 2025. Aman gave me a crash course on how to build this critical skill in our interview.

 

Timestamps:

  • (00:00) Evals force you to get into your user's shoes

  • (03:42) 5 skills to build right now to become a great AI PM

  • (07:43) How curiosity leads to better AI products

  • (10:20) You can build AI without the job title

  • (17:00) A deep dive into why AI evals are so important

  • (24:53) Example of running evals for an AI customer support agent

  • (31:32) When humans need to be in the loop for evals

  • (35:46) How to get better at writing evals right now

  • (39:39) My personal prompt and eval for transcripts

  • (44:23) Catching the AI wave is alot like surfing

     

    Get the takeaways: https://creatoreconomy.so/p/the-ai-skill-that-will-define-your-pm-career-aman-khan

     

    Where to find Aman:

    LinkedIn: https://www.linkedin.com/in/amanberkeley/

    Arize: https://arize.com/


    📌 Subscribe to this channel – more interviews coming soon!

    ...more
    View all episodesView all episodes
    Download on the App Store

    Behind the CraftBy Peter Yang

    • 5
    • 5
    • 5
    • 5
    • 5

    5

    7 ratings


    More shows like Behind the Craft

    View all
    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

    546 Listeners

    The a16z Show by Andreessen Horowitz

    The a16z Show

    1,102 Listeners

    Masters of Scale by WaitWhat

    Masters of Scale

    3,980 Listeners

    Y Combinator Startup Podcast by Y Combinator

    Y Combinator Startup Podcast

    235 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    212 Listeners

    My First Million by Hubspot Media

    My First Million

    2,663 Listeners

    Big Technology Podcast by Alex Kantrowitz

    Big Technology Podcast

    512 Listeners

    The Startup Ideas Podcast by Greg Isenberg

    The Startup Ideas Podcast

    212 Listeners

    Latent Space: The AI Engineer Podcast by Latent.Space

    Latent Space: The AI Engineer Podcast

    102 Listeners

    The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

    The AI Daily Brief: Artificial Intelligence News and Analysis

    680 Listeners

    Everyday AI Podcast – An AI and ChatGPT Podcast by Everyday AI

    Everyday AI Podcast – An AI and ChatGPT Podcast

    112 Listeners

    AI + a16z by a16z

    AI + a16z

    33 Listeners

    AI Explored by Michael Stelzner, Social Media Examiner—AI marketing

    AI Explored

    97 Listeners

    Uncapped with Jack Altman by Alt Capital

    Uncapped with Jack Altman

    41 Listeners

    How I AI by Claire Vo

    How I AI

    161 Listeners