Training Data

Databricks Founder Ion Stoica: Turning Academic Open Source into Startup Success


Listen Later

Berkeley professor Ion Stoica, co-founder of Databricks and Anyscale, transformed the open source projects Spark and Ray into successful AI infrastructure companies. He talks about what mattered most for Databricks' success -- the focus on making Spark win and making Databricks the best place to run Spark. He highlights the importance of striking key partnerships -- the Microsoft partnership in particular that accelerated Databricks' growth and contributed to Spark's dominance among data scientists and AI engineers. He also shares his perspective on finding new problems to work on, which holds lessons for aspiring founders and builders: 1) building systems in new areas that, if widely adopted, put you in the best position to understand the new problem space, and 2) focusing on a problem that is more important tomorrow than today.


Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital


Mentioned in this episode: 

  • Spark: The open source platform for data engineering that Databricks was originally based on.
  • Ray: Open source framework to manage, executes and optimizes compute needs across AI workloads, now productized through Anyscale
  • MosaicML: Generative AI startups founded by Naveen Rao that Databricks acquired in 2023.
  • Unity Catalog: Data and AI governance solution from Databricks.
  • CIB Berkeley: Multi-strategy hedge fund at UC Berkeley that commercializes research in the UC system.
  • Hadoop: A long-time leading platform for large scale distributed computing.
  • VLLM and Chatbot Arena: Two of Ion’s students’ projects that he wanted to highlight.
    ...more
    View all episodesView all episodes
    Download on the App Store

    Training DataBy Sequoia Capital

    • 4.3
    • 4.3
    • 4.3
    • 4.3
    • 4.3

    4.3

    31 ratings


    More shows like Training Data

    View all
    This Week in Startups by Jason Calacanis

    This Week in Startups

    1,268 Listeners

    a16z Podcast by Andreessen Horowitz

    a16z Podcast

    1,003 Listeners

    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

    513 Listeners

    Invest Like the Best with Patrick O'Shaughnessy by Colossus | Investing & Business Podcasts

    Invest Like the Best with Patrick O'Shaughnessy

    2,294 Listeners

    Y Combinator Startup Podcast by Y Combinator

    Y Combinator Startup Podcast

    209 Listeners

    Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

    Machine Learning Street Talk (MLST)

    88 Listeners

    Grit by Kleiner Perkins

    Grit

    190 Listeners

    Dwarkesh Podcast by Dwarkesh Patel

    Dwarkesh Podcast

    343 Listeners

    The Logan Bartlett Show by by Redpoint Ventures

    The Logan Bartlett Show

    190 Listeners

    No Priors: Artificial Intelligence | Technology | Startups by Conviction

    No Priors: Artificial Intelligence | Technology | Startups

    125 Listeners

    Latent Space: The AI Engineer Podcast by swyx + Alessio

    Latent Space: The AI Engineer Podcast

    63 Listeners

    Crucible Moments by Sequoia Capital

    Crucible Moments

    90 Listeners

    The Ben & Marc Show by Marc Andreessen, Ben Horowitz

    The Ben & Marc Show

    120 Listeners

    BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

    BG2Pod with Brad Gerstner and Bill Gurley

    438 Listeners

    AI + a16z by a16z

    AI + a16z

    29 Listeners

    Lightcone Podcast by Y Combinator

    Lightcone Podcast

    19 Listeners