Vanishing Gradients

Episode 45: Your AI application is broken. Here’s what to do about it.


Listen Later

Too many teams are building AI applications without truly understanding why their models fail. Instead of jumping straight to LLM evaluations, dashboards, or vibe checks, how do you actually fix a broken AI app?

In this episode, Hugo speaks with Hamel Husain, longtime ML engineer, open-source contributor, and consultant, about why debugging generative AI systems starts with looking at your data.

In this episode, we dive into:

  • Why “look at your data” is the best debugging advice no one follows.
  • How spreadsheet-based error analysis can uncover failure modes faster than complex dashboards.
  • The role of synthetic data in bootstrapping evaluation.
  • When to trust LLM judges—and when they’re misleading.
  • Why most AI dashboards measuring truthfulness, helpfulness, and conciseness are often a waste of time.
  • If you're building AI-powered applications, this episode will change how you approach debugging, iteration, and improving model performance in production.

    LINKS

    • The podcast livestream on YouTube
    • Hamel's blog
    • Hamel on twitter
    • Hugo on twitter
    • Vanishing Gradients on twitter
    • Vanishing Gradients on YouTube
    • Vanishing Gradients on Twitter
    • Vanishing Gradients on Lu.ma

    • Building LLM Application for Data Scientists and SWEs, Hugo course on Maven (use VG25 code for 25% off)

    • Hugo is also running a free lightning lesson next week on LLM Agents: When to Use Them (and When Not To)

    • ...more
      View all episodesView all episodes
      Download on the App Store

      Vanishing GradientsBy Hugo Bowne-Anderson

      • 5
      • 5
      • 5
      • 5
      • 5

      5

      11 ratings


      More shows like Vanishing Gradients

      View all
      a16z Podcast by Andreessen Horowitz

      a16z Podcast

      1,001 Listeners

      Data Skeptic by Kyle Polich

      Data Skeptic

      470 Listeners

      Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

      Super Data Science: ML & AI Podcast with Jon Krohn

      296 Listeners

      DataFramed by DataCamp

      DataFramed

      269 Listeners

      Practical AI by Practical AI LLC

      Practical AI

      190 Listeners

      Last Week in AI by Skynet Today

      Last Week in AI

      281 Listeners

      Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

      Machine Learning Street Talk (MLST)

      88 Listeners

      Dwarkesh Podcast by Dwarkesh Patel

      Dwarkesh Podcast

      354 Listeners

      No Priors: Artificial Intelligence | Technology | Startups by Conviction

      No Priors: Artificial Intelligence | Technology | Startups

      125 Listeners

      This Day in AI Podcast by Michael Sharkey, Chris Sharkey

      This Day in AI Podcast

      190 Listeners

      Latent Space: The AI Engineer Podcast by swyx + Alessio

      Latent Space: The AI Engineer Podcast

      63 Listeners

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

      424 Listeners

      The Next Wave - AI and The Future of Technology by Hubspot Media

      The Next Wave - AI and The Future of Technology

      57 Listeners

      Training Data by Sequoia Capital

      Training Data

      36 Listeners

      High Signal: Data Science | Career | AI by Delphina

      High Signal: Data Science | Career | AI

      4 Listeners