Latent Space: The AI Engineer Podcast

After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs


Listen Later

Fei-Fei Li and Justin Johnson are cofounders of World Labs, who have recently launched Marble (https://marble.worldlabs.ai/), a new kind of generative “world model” that can create editable 3D environments from text, images, and other spatial inputs. Marble lets creators generate persistent 3D worlds, precisely control cameras, and interactively edit scenes, making it a powerful tool for games, film, VR, robotics simulation, and more. In this episode, Fei-Fei and Justin share how their journey from ImageNet and Stanford research led to World Labs, why spatial intelligence is the next frontier after LLMs, and how world models could change how machines see, understand, and build in 3D.

We discuss:

  • The massive compute scaling from AlexNet to today and why world models and spatial data are the most compelling way to “soak up” modern GPU clusters compared to language alone.

  • What Marble actually is: a generative model of 3D worlds that turns text and images into editable scenes using Gaussian splats, supports precise camera control and recording, and runs interactively on phones, laptops, and VR headsets.

  • Fei-fei’s essay (https://drfeifei.substack.com/p/from-words-to-worlds-spatial-intelligence) on spatial intelligence as a distinct form of intelligence from language: from picking up a mug to inferring the 3D structure of DNA, and why language is a lossy, low-bandwidth channel for describing the rich 3D/4D world we live in.

  • Whether current models “understand” physics or just fit patterns: the gap between predicting orbits and discovering F=ma, and how attaching physical properties to splats and distilling physics engines into neural networks could lead to genuine causal reasoning.

  • The changing role of academia in AI, why Fei-Fei worries more about under-resourced universities than “open vs closed,” and how initiatives like national AI compute clouds and open benchmarks can rebalance the ecosystem.

  • Why transformers are fundamentally set models, not sequence models, and how that perspective opens up new architectures for world models, especially as hardware shifts from single GPUs to massive distributed clusters.

  • Real use cases for Marble today: previsualization and VFX, game environments, virtual production, interior and architectural design (including kitchen remodels), and generating synthetic simulation worlds for training embodied agents and robots.

  • How spatial intelligence and language intelligence will work together in multimodal systems, and why the goal isn’t to throw away LLMs but to complement them with rich, embodied models of the world.

  • Fei-Fei and Justin’s long-term vision for spatial intelligence: from creative tools for artists and game devs to broader applications in science, medicine, and real-world decision-making.

Fei-Fei Li

  • X: https://x.com/drfeifei

  • LinkedIn: https://www.linkedin.com/in/fei-fei-li-4541247

Justin Johnson

  • X: https://x.com/jcjohnss

  • LinkedIn: https://www.linkedin.com/in/justin-johnson-41b43664

Where to find Latent Space

  • X: https://x.com/latentspacepod

  • Substack: https://www.latent.space/

Chapters
  • 00:00:00 Introduction and the Fei-Fei Li & Justin Johnson Partnership
  • 00:02:00 From ImageNet to World Models: The Evolution of Computer Vision
  • 00:12:42 Dense Captioning and Early Vision-Language Work
  • 00:19:57 Spatial Intelligence: Beyond Language Models
  • 00:28:46 Introducing Marble: World Labs' First Spatial Intelligence Model
  • 00:33:21 Gaussian Splats and the Technical Architecture of Marble
  • 00:22:10 Physics, Dynamics, and the Future of World Models
  • 00:41:09 Multimodality and the Interplay of Language and Space
  • 00:37:37 Use Cases: From Creative Industries to Robotics and Embodied AI
  • 00:56:58 Hiring, Research Directions, and the Future of World Labs

...more
View all episodesView all episodes
Download on the App Store

Latent Space: The AI Engineer PodcastBy swyx + Alessio

  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

86 ratings


More shows like Latent Space: The AI Engineer Podcast

View all
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

535 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

288 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

1,094 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

303 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

336 Listeners

Y Combinator Startup Podcast by Y Combinator

Y Combinator Startup Podcast

228 Listeners

Practical AI by Practical AI LLC

Practical AI

210 Listeners

Google DeepMind: The Podcast by Hannah Fry

Google DeepMind: The Podcast

198 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

507 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

134 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

210 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

598 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

511 Listeners

Lightcone Podcast by Y Combinator

Lightcone Podcast

23 Listeners

Training Data by Sequoia Capital

Training Data

38 Listeners