Vanishing Gradients

Episode 29: Lessons from a Year of Building with LLMs (Part 1)


Listen Later

Hugo speaks about Lessons Learned from a Year of Building with LLMs with Eugene Yan from Amazon, Bryan Bischof from Hex, Charles Frye from Modal, Hamel Husain from Parlance Labs, and Shreya Shankar from UC Berkeley.

These five guests, along with Jason Liu who couldn't join us, have spent the past year building real-world applications with Large Language Models (LLMs). They've distilled their experiences into a report of 42 lessons across operational, strategic, and tactical dimensions, and they're here to share their insights.

We’ve split this roundtable into 2 episodes and, in this first episode, we'll explore:

  • The critical role of evaluation and monitoring in LLM applications and why they're non-negotiable, including "evals" - short for evaluations, which are automated tests for assessing LLM performance and output quality;
  • Why data literacy is your secret weapon in the AI landscape;
  • The fine-tuning dilemma: when to do it and when to skip it;
  • Real-world lessons from building LLM applications that textbooks won't teach you;
  • The evolving role of data scientists and AI engineers in the age of AI.
  • Although we're focusing on LLMs, many of these insights apply broadly to data science, machine learning, and product development, more generally.

    LINKS

    • The livestream on YouTube
    • The Report: What We’ve Learned From A Year of Building with LLMs
    • About the Guests/Authors <-- connect with them all on LinkedIn, follow them on Twitter, subscribe to their newsletters! (Seriously, though, the amount of collective wisdom here is 🤑
    • Your AI product needs evals by Hamel Husain
    • Prompting Fundamentals and How to Apply them Effectively by Eugene Yan
    • Fuck You, Show Me The Prompt by Hamel Husain
    • Vanishing Gradients on YouTube
    • Vanishing Gradients on Twitter
    • Vanishing Gradients on Lu.ma
    • ...more
      View all episodesView all episodes
      Download on the App Store

      Vanishing GradientsBy Hugo Bowne-Anderson

      • 5
      • 5
      • 5
      • 5
      • 5

      5

      11 ratings


      More shows like Vanishing Gradients

      View all
      a16z Podcast by Andreessen Horowitz

      a16z Podcast

      1,001 Listeners

      Data Skeptic by Kyle Polich

      Data Skeptic

      470 Listeners

      Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

      Super Data Science: ML & AI Podcast with Jon Krohn

      296 Listeners

      DataFramed by DataCamp

      DataFramed

      269 Listeners

      Practical AI by Practical AI LLC

      Practical AI

      190 Listeners

      Last Week in AI by Skynet Today

      Last Week in AI

      281 Listeners

      Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

      Machine Learning Street Talk (MLST)

      88 Listeners

      Dwarkesh Podcast by Dwarkesh Patel

      Dwarkesh Podcast

      354 Listeners

      No Priors: Artificial Intelligence | Technology | Startups by Conviction

      No Priors: Artificial Intelligence | Technology | Startups

      125 Listeners

      This Day in AI Podcast by Michael Sharkey, Chris Sharkey

      This Day in AI Podcast

      190 Listeners

      Latent Space: The AI Engineer Podcast by swyx + Alessio

      Latent Space: The AI Engineer Podcast

      63 Listeners

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

      The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

      424 Listeners

      The Next Wave - AI and The Future of Technology by Hubspot Media

      The Next Wave - AI and The Future of Technology

      57 Listeners

      Training Data by Sequoia Capital

      Training Data

      36 Listeners

      High Signal: Data Science | Career | AI by Delphina

      High Signal: Data Science | Career | AI

      4 Listeners