Sign up to save your podcasts
Or
Share When LeJEPA Truly Learns a World Model
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
When LeJEPA Truly Learns a World Model
This episode explores the paper When Does LeJEPA Learn a World Model? and uses it to examine what should count as a genuine world model in latent predictive learning, contrasting JEPA-style representation prediction with generative reconstruction. It explains why good probe scores are not enough: the real standard is linear identifiability, where a single global linear map recovers the environment’s hidden state well enough to support planning and compositional generalization. The discussion centers on the paper’s main theorem that, under stationary additive-noise dynamics with Gaussian latent variables, LeJEPA’s alignment objective plus SIGReg recovers the true latent state up to an orthogonal rotation, and on the sharper converse result that this universal guarantee fails for non-Gaussian latents. Listeners get a rigorous argument for when latent models are truly learning the world’s coordinates instead of merely extracting features that happen to be useful on downstream tasks.
Sources:
1. When LeJEPA Truly Learns a World Model
https://arxiv.org/pdf/2605.26379
2. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations — Francesco Locatello, Stefan Bauer, Mario Lucic, et al., 2018
https://scholar.google.com/scholar?q=Challenging+Common+Assumptions+in+the+Unsupervised+Learning+of+Disentangled+Representations
3. Variational Autoencoders and Nonlinear ICA: A Unifying Framework — Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, Aapo Hyvarinen, 2019
https://scholar.google.com/scholar?q=Variational+Autoencoders+and+Nonlinear+ICA%3A+A+Unifying+Framework
4. On Linear Identifiability of Learned Representations — Geoffrey Roeder, Luke Metz, Diederik P. Kingma, 2020
https://scholar.google.com/scholar?q=On+Linear+Identifiability+of+Learned+Representations
5. Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning — Aapo Hyvarinen, Ilyes Khemakhem, Hiroshi Morioka, 2023
https://scholar.google.com/scholar?q=Nonlinear+Independent+Component+Analysis+for+Principled+Disentanglement+in+Unsupervised+Deep+Learning
6. Auto-Encoding Variational Bayes — Diederik P. Kingma, Max Welling, 2013
https://scholar.google.com/scholar?q=Auto-Encoding+Variational+Bayes
7. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning — Adrien Bardes, Jean Ponce, Yann LeCun, 2021
https://scholar.google.com/scholar?q=VICReg%3A+Variance-Invariance-Covariance+Regularization+for+Self-Supervised+Learning
8. LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics — Randall Balestriero, Yann LeCun, 2025
https://scholar.google.com/scholar?q=LeJEPA%3A+Provable+and+Scalable+Self-Supervised+Learning+Without+the+Heuristics
9. When Does LeJEPA Learn a World Model? — David Klindt, Yann LeCun, Randall Balestriero, 2026
https://scholar.google.com/scholar?q=When+Does+LeJEPA+Learn+a+World+Model%3F
10. LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels — Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, Randall Balestriero, 2026
https://scholar.google.com/scholar?q=LeWorldModel%3A+Stable+End-to-End+Joint-Embedding+Predictive+Architecture+from+Pixels
11. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — Mido Assran et al., 2025
https://scholar.google.com/scholar?q=V-JEPA+2%3A+Self-Supervised+Video+Models+Enable+Understanding%2C+Prediction+and+Planning
12. Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning — Aapo Hyvarinen, Hiroaki Sasaki, Richard E. Turner, 2018
https://scholar.google.com/scholar?q=Nonlinear+ICA+Using+Auxiliary+Variables+and+Generalized+Contrastive+Learning
13. Joint Embedding Predictive Architectures Focus on Slow Features — Vlad Sobal, Jyothir S V, Siddhartha Jalagam, Nicolas Carion, Kyunghyun Cho, Yann LeCun, 2022
https://scholar.google.com/scholar?q=Joint+Embedding+Predictive+Architectures+Focus+on+Slow+Features
14. Cross-Entropy Is All You Need To Invert the Data Generating Process — Patrik Reizinger, Alice Bizeul, Attila Juhos, Julia E. Vogt, Randall Balestriero, Wieland Brendel, David Klindt, 2024
https://scholar.google.com/scholar?q=Cross-Entropy+Is+All+You+Need+To+Invert+the+Data+Generating+Process
15. Identifiability of latent-variable and structural-equation models: from linear to nonlinear — Aapo Hyvarinen, Ilyes Khemakhem, Ricardo Monti, 2023
https://scholar.google.com/scholar?q=Identifiability+of+latent-variable+and+structural-equation+models%3A+from+linear+to+nonlinear
16. On the Identifiability of Sparse ICA without Assuming Non-Gaussianity — Ignavier Ng, Yujia Zheng, Xinshuai Dong, Kun Zhang, 2024
https://scholar.google.com/scholar?q=On+the+Identifiability+of+Sparse+ICA+without+Assuming+Non-Gaussianity
17. Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity — Emiliyan Gospodinov, Vaisakh Shaj, Philipp Becker, Stefan Geyer, Gerhard Neumann, 2024
https://scholar.google.com/scholar?q=Adaptive+World+Models%3A+Learning+Behaviors+by+Latent+Imagination+Under+Non-Stationarity
18. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors — Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long, 2023
https://scholar.google.com/scholar?q=Koopa%3A+Learning+Non-stationary+Time+Series+Dynamics+with+Koopman+Predictors
19. Simplifying Latent Dynamics with Softly State-Invariant World Models — Tankred Saanum, Peter Dayan, Eric Schulz, 2024
https://scholar.google.com/scholar?q=Simplifying+Latent+Dynamics+with+Softly+State-Invariant+World+Models
20. Structured World Models from Human Videos — Russell Mendonca, Shikhar Bahl, Deepak Pathak, 2023
https://scholar.google.com/scholar?q=Structured+World+Models+from+Human+Videos
21. AI Post Transformers: LeWorldModel: Stable Joint-Embedding World Models from Pixels — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-03-25-leworldmodel-stable-joint-embedding-worl-650f9f.mp3
22. AI Post Transformers: Causal-JEPA for Object-Level World Models — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-15-causal-jepa-for-object-level-world-model-311a8b.mp3
23. AI Post Transformers: Learning Latent Action World Models from Video — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-09-learning-latent-action-world-models-from-1570a4.mp3
Interactive Visualization: When LeJEPA Truly Learns a World Model
View all episodes
By mcgrofThis episode explores the paper When Does LeJEPA Learn a World Model? and uses it to examine what should count as a genuine world model in latent predictive learning, contrasting JEPA-style representation prediction with generative reconstruction. It explains why good probe scores are not enough: the real standard is linear identifiability, where a single global linear map recovers the environment’s hidden state well enough to support planning and compositional generalization. The discussion centers on the paper’s main theorem that, under stationary additive-noise dynamics with Gaussian latent variables, LeJEPA’s alignment objective plus SIGReg recovers the true latent state up to an orthogonal rotation, and on the sharper converse result that this universal guarantee fails for non-Gaussian latents. Listeners get a rigorous argument for when latent models are truly learning the world’s coordinates instead of merely extracting features that happen to be useful on downstream tasks.
Sources:
1. When LeJEPA Truly Learns a World Model
https://arxiv.org/pdf/2605.26379
2. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations — Francesco Locatello, Stefan Bauer, Mario Lucic, et al., 2018
https://scholar.google.com/scholar?q=Challenging+Common+Assumptions+in+the+Unsupervised+Learning+of+Disentangled+Representations
3. Variational Autoencoders and Nonlinear ICA: A Unifying Framework — Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, Aapo Hyvarinen, 2019
https://scholar.google.com/scholar?q=Variational+Autoencoders+and+Nonlinear+ICA%3A+A+Unifying+Framework
4. On Linear Identifiability of Learned Representations — Geoffrey Roeder, Luke Metz, Diederik P. Kingma, 2020
https://scholar.google.com/scholar?q=On+Linear+Identifiability+of+Learned+Representations
5. Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning — Aapo Hyvarinen, Ilyes Khemakhem, Hiroshi Morioka, 2023
https://scholar.google.com/scholar?q=Nonlinear+Independent+Component+Analysis+for+Principled+Disentanglement+in+Unsupervised+Deep+Learning
6. Auto-Encoding Variational Bayes — Diederik P. Kingma, Max Welling, 2013
https://scholar.google.com/scholar?q=Auto-Encoding+Variational+Bayes
7. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning — Adrien Bardes, Jean Ponce, Yann LeCun, 2021
https://scholar.google.com/scholar?q=VICReg%3A+Variance-Invariance-Covariance+Regularization+for+Self-Supervised+Learning
8. LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics — Randall Balestriero, Yann LeCun, 2025
https://scholar.google.com/scholar?q=LeJEPA%3A+Provable+and+Scalable+Self-Supervised+Learning+Without+the+Heuristics
9. When Does LeJEPA Learn a World Model? — David Klindt, Yann LeCun, Randall Balestriero, 2026
https://scholar.google.com/scholar?q=When+Does+LeJEPA+Learn+a+World+Model%3F
10. LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels — Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, Randall Balestriero, 2026
https://scholar.google.com/scholar?q=LeWorldModel%3A+Stable+End-to-End+Joint-Embedding+Predictive+Architecture+from+Pixels
11. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — Mido Assran et al., 2025
https://scholar.google.com/scholar?q=V-JEPA+2%3A+Self-Supervised+Video+Models+Enable+Understanding%2C+Prediction+and+Planning
12. Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning — Aapo Hyvarinen, Hiroaki Sasaki, Richard E. Turner, 2018
https://scholar.google.com/scholar?q=Nonlinear+ICA+Using+Auxiliary+Variables+and+Generalized+Contrastive+Learning
13. Joint Embedding Predictive Architectures Focus on Slow Features — Vlad Sobal, Jyothir S V, Siddhartha Jalagam, Nicolas Carion, Kyunghyun Cho, Yann LeCun, 2022
https://scholar.google.com/scholar?q=Joint+Embedding+Predictive+Architectures+Focus+on+Slow+Features
14. Cross-Entropy Is All You Need To Invert the Data Generating Process — Patrik Reizinger, Alice Bizeul, Attila Juhos, Julia E. Vogt, Randall Balestriero, Wieland Brendel, David Klindt, 2024
https://scholar.google.com/scholar?q=Cross-Entropy+Is+All+You+Need+To+Invert+the+Data+Generating+Process
15. Identifiability of latent-variable and structural-equation models: from linear to nonlinear — Aapo Hyvarinen, Ilyes Khemakhem, Ricardo Monti, 2023
https://scholar.google.com/scholar?q=Identifiability+of+latent-variable+and+structural-equation+models%3A+from+linear+to+nonlinear
16. On the Identifiability of Sparse ICA without Assuming Non-Gaussianity — Ignavier Ng, Yujia Zheng, Xinshuai Dong, Kun Zhang, 2024
https://scholar.google.com/scholar?q=On+the+Identifiability+of+Sparse+ICA+without+Assuming+Non-Gaussianity
17. Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity — Emiliyan Gospodinov, Vaisakh Shaj, Philipp Becker, Stefan Geyer, Gerhard Neumann, 2024
https://scholar.google.com/scholar?q=Adaptive+World+Models%3A+Learning+Behaviors+by+Latent+Imagination+Under+Non-Stationarity
18. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors — Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long, 2023
https://scholar.google.com/scholar?q=Koopa%3A+Learning+Non-stationary+Time+Series+Dynamics+with+Koopman+Predictors
19. Simplifying Latent Dynamics with Softly State-Invariant World Models — Tankred Saanum, Peter Dayan, Eric Schulz, 2024
https://scholar.google.com/scholar?q=Simplifying+Latent+Dynamics+with+Softly+State-Invariant+World+Models
20. Structured World Models from Human Videos — Russell Mendonca, Shikhar Bahl, Deepak Pathak, 2023
https://scholar.google.com/scholar?q=Structured+World+Models+from+Human+Videos
21. AI Post Transformers: LeWorldModel: Stable Joint-Embedding World Models from Pixels — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-03-25-leworldmodel-stable-joint-embedding-worl-650f9f.mp3
22. AI Post Transformers: Causal-JEPA for Object-Level World Models — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-15-causal-jepa-for-object-level-world-model-311a8b.mp3
23. AI Post Transformers: Learning Latent Action World Models from Video — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-09-learning-latent-action-world-models-from-1570a4.mp3
Interactive Visualization: When LeJEPA Truly Learns a World Model