Training Data

Why Vlad Tenev and Tudor Achim of Harmonic Think AI Is About to Change Math—and Why It Matters


Listen Later

Adding code to LLM training data is a known method of improving a model’s reasoning skills. But wouldn’t math, the basis of all reasoning, be even better? Up until recently, there just wasn’t enough usable data that describes mathematics to make this feasible.

A few years ago, Vlad Tenev (also founder of Robinhood) and Tudor Achim noticed the rise of the community around an esoteric programming language called Lean that was gaining traction among mathematicians. The combination of that and the past decade’s rise of autoregressive models capable of fast, flexible learning made them think the time was now and they founded Harmonic. Their mission is both lofty—mathematical superintelligence—and imminently practical, verifying all safety-critical software.

Hosted by: Sonya Huang and Pat Grady, Sequoia Capital 

Mentioned in this episode:

  • IMO and the Millennium Prize: Two significant global competitions Harmonic hopes to win (soon)
  • Riemann hypothesis: One of the most difficult unsolved math conjectures (and a Millenium Prize problem) most recently in the sights of MIT mathematician Larry Guth
  • Terry Tao: perhaps the greatest living mathematician and Vlad’s professor at UCLA
  • Lean: an open source functional language for code verification launched by Leonardo de Moura when at Microsoft Research in 2013 that powers the Lean Theorem Prover
  • mathlib: the largest math textbook in the world, all written in Lean
  • Metaculus: online prediction platform that tracks and scores thousands of forecasters
  • Minecraft Beaten in 20 Seconds: The video Vlad references as an analogy to AI math
  • Navier-Stokes equations: another important Millenium Prize math problem. Vlad considers this more tractable that Riemann
  • John von Neumann: Hungarian mathematician and polymath that made foundational contributions to computing, the Manhattan Project and game theory
  • Gottfried Wilhelm Leibniz: co-inventor of calculus and (remarkably) creator of the “universal characteristic,” a system for reasoning through a language of symbols and calculations—anticipating Lean and Harmonic by 350 years!


    00:00 - Introduction

    01:42 - Math is reasoning

    06:16 - Studying with the world's greatest living mathematician

    10:18 - What does the math community think of AI math?

    15:11 - Recursive self-improvement

    18:31 - What is Lean?

    21:05 - Why now?

    22:46 - Synthetic data is the fuel for the model

    27:29 - How fast will your model get better?

    29:45 - Exploring the frontiers of human knowledge

    34:11 - Lightning round

    ...more
    View all episodesView all episodes
    Download on the App Store

    Training DataBy Sequoia Capital

    • 4.2
    • 4.2
    • 4.2
    • 4.2
    • 4.2

    4.2

    36 ratings


    More shows like Training Data

    View all
    This Week in Startups by Jason Calacanis

    This Week in Startups

    1,281 Listeners

    a16z Podcast by Andreessen Horowitz

    a16z Podcast

    1,081 Listeners

    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

    The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

    525 Listeners

    Y Combinator Startup Podcast by Y Combinator

    Y Combinator Startup Podcast

    221 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    206 Listeners

    Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

    Machine Learning Street Talk (MLST)

    88 Listeners

    Grit by Kleiner Perkins

    Grit

    189 Listeners

    Dwarkesh Podcast by Dwarkesh Patel

    Dwarkesh Podcast

    456 Listeners

    No Priors: Artificial Intelligence | Technology | Startups by Conviction

    No Priors: Artificial Intelligence | Technology | Startups

    131 Listeners

    Latent Space: The AI Engineer Podcast by swyx + Alessio

    Latent Space: The AI Engineer Podcast

    96 Listeners

    Crucible Moments by Sequoia Capital

    Crucible Moments

    91 Listeners

    BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

    BG2Pod with Brad Gerstner and Bill Gurley

    483 Listeners

    AI + a16z by a16z

    AI + a16z

    31 Listeners

    Lightcone Podcast by Y Combinator

    Lightcone Podcast

    17 Listeners

    Uncapped with Jack Altman by Alt Capital

    Uncapped with Jack Altman

    41 Listeners

    Cheeky Pint by Stripe

    Cheeky Pint

    17 Listeners