Sign up to save your podcasts
Or
Share Episode 6: The Latent Canvas Deconstructing Ha & Schmidhuber's Masterpiece
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Episode 6: The Latent Canvas Deconstructing Ha & Schmidhuber's Masterpiece
Episode 6 of The World Model Podcast takes us back to one of the true turning points in modern AI: the 2018 paper “World Models” by David Ha and Jürgen Schmidhuber. While not the first to explore the idea, this paper distilled the concept of world models into a form so simple, so elegant, and so powerful that it became a lighthouse for an entire generation of researchers.In this episode, we break down the architecture that made the paper revolutionary. You’ll learn how Ha & Schmidhuber separated intelligence into two core components:
• The Vision Model (V): a variational autoencoder that compresses raw sensory input into a compact latent representation—the AI’s distilled “now.”
• The Memory Model (M): a recurrent mixture-density network that predicts the evolution of those latent states, learning the dynamics of the world itself.We explore the remarkable insight that emerges when you close the loop between these two systems. Once trained, the AI can detach from reality and simulate entire trajectories within its own mind dreaming full sequences of imagined futures, practicing inside a world made of its own latent representations.Through the paper’s iconic car-racing experiment, we show how the AI learned not by seeing the game, but by imagining it running entire laps within its internal simulation. This approach became a foundational blueprint for DeepMind’s later Dreamer architectures and a defining milestone in model-based reinforcement learning.The episode culminates in a bold claim: that World Models may ultimately be remembered as more profound than Attention Is All You Need. The Transformer gave us scale, fluency, and the era of LLMs. But World Models offered something deeper a blueprint for machines that construct, reason about, and experiment within their own internal realities.Next episode, we take this blueprint into the real world, examining how world-model architectures are shaping the intelligence inside a Tesla as it navigates our chaotic, unpredictable environment.If you want to understand the intellectual roots of the world-model revolution, this is the episode to start with.
Become a supporter of this podcast: https://www.spreaker.com/podcast/the-world-model-podcast--6814682/support.
• The Vision Model (V): a variational autoencoder that compresses raw sensory input into a compact latent representation—the AI’s distilled “now.”
• The Memory Model (M): a recurrent mixture-density network that predicts the evolution of those latent states, learning the dynamics of the world itself.We explore the remarkable insight that emerges when you close the loop between these two systems. Once trained, the AI can detach from reality and simulate entire trajectories within its own mind dreaming full sequences of imagined futures, practicing inside a world made of its own latent representations.Through the paper’s iconic car-racing experiment, we show how the AI learned not by seeing the game, but by imagining it running entire laps within its internal simulation. This approach became a foundational blueprint for DeepMind’s later Dreamer architectures and a defining milestone in model-based reinforcement learning.The episode culminates in a bold claim: that World Models may ultimately be remembered as more profound than Attention Is All You Need. The Transformer gave us scale, fluency, and the era of LLMs. But World Models offered something deeper a blueprint for machines that construct, reason about, and experiment within their own internal realities.Next episode, we take this blueprint into the real world, examining how world-model architectures are shaping the intelligence inside a Tesla as it navigates our chaotic, unpredictable environment.If you want to understand the intellectual roots of the world-model revolution, this is the episode to start with.
Become a supporter of this podcast: https://www.spreaker.com/podcast/the-world-model-podcast--6814682/support.
View all episodes
By World ModelsEpisode 6 of The World Model Podcast takes us back to one of the true turning points in modern AI: the 2018 paper “World Models” by David Ha and Jürgen Schmidhuber. While not the first to explore the idea, this paper distilled the concept of world models into a form so simple, so elegant, and so powerful that it became a lighthouse for an entire generation of researchers.In this episode, we break down the architecture that made the paper revolutionary. You’ll learn how Ha & Schmidhuber separated intelligence into two core components:
• The Vision Model (V): a variational autoencoder that compresses raw sensory input into a compact latent representation—the AI’s distilled “now.”
• The Memory Model (M): a recurrent mixture-density network that predicts the evolution of those latent states, learning the dynamics of the world itself.We explore the remarkable insight that emerges when you close the loop between these two systems. Once trained, the AI can detach from reality and simulate entire trajectories within its own mind dreaming full sequences of imagined futures, practicing inside a world made of its own latent representations.Through the paper’s iconic car-racing experiment, we show how the AI learned not by seeing the game, but by imagining it running entire laps within its internal simulation. This approach became a foundational blueprint for DeepMind’s later Dreamer architectures and a defining milestone in model-based reinforcement learning.The episode culminates in a bold claim: that World Models may ultimately be remembered as more profound than Attention Is All You Need. The Transformer gave us scale, fluency, and the era of LLMs. But World Models offered something deeper a blueprint for machines that construct, reason about, and experiment within their own internal realities.Next episode, we take this blueprint into the real world, examining how world-model architectures are shaping the intelligence inside a Tesla as it navigates our chaotic, unpredictable environment.If you want to understand the intellectual roots of the world-model revolution, this is the episode to start with.
Become a supporter of this podcast: https://www.spreaker.com/podcast/the-world-model-podcast--6814682/support.
• The Vision Model (V): a variational autoencoder that compresses raw sensory input into a compact latent representation—the AI’s distilled “now.”
• The Memory Model (M): a recurrent mixture-density network that predicts the evolution of those latent states, learning the dynamics of the world itself.We explore the remarkable insight that emerges when you close the loop between these two systems. Once trained, the AI can detach from reality and simulate entire trajectories within its own mind dreaming full sequences of imagined futures, practicing inside a world made of its own latent representations.Through the paper’s iconic car-racing experiment, we show how the AI learned not by seeing the game, but by imagining it running entire laps within its internal simulation. This approach became a foundational blueprint for DeepMind’s later Dreamer architectures and a defining milestone in model-based reinforcement learning.The episode culminates in a bold claim: that World Models may ultimately be remembered as more profound than Attention Is All You Need. The Transformer gave us scale, fluency, and the era of LLMs. But World Models offered something deeper a blueprint for machines that construct, reason about, and experiment within their own internal realities.Next episode, we take this blueprint into the real world, examining how world-model architectures are shaping the intelligence inside a Tesla as it navigates our chaotic, unpredictable environment.If you want to understand the intellectual roots of the world-model revolution, this is the episode to start with.
Become a supporter of this podcast: https://www.spreaker.com/podcast/the-world-model-podcast--6814682/support.