Embodied AI 101

Embodied AI 101

By Shaoqing TanTechnology
Download on the App Store

Embodied AI 101 episodes

  • Episode 33: How Meta Trained SAM‑3D: Extending “Segment Anything” to 3D
    **Meta’s SAM‑3D** is a new **foundation model for 3D** vision that extends the 2D **Segment Anything Model (SAM)** into the third dimension. It actually consists of two specialized models – **SAM‑3D Objects** (for general 3D object and scene reconstruction) and **SAM‑3D Body** (for full 3D human pose and shape estimation). Both were trained on unprecedented datasets using novel training pipelines. In this post, we’ll dive into **how SAM‑3D was trained** – covering the datasets (real vs synthe...
    58 min
  • Episode 32: Closing the Sim-to-Real Loop: GSWorld’s Photorealistic Real2Sim2Real Pipeline
    ## Motivation: Why Sim-to-Real Matters Now
    Robotic manipulation and navigation systems are reaching levels of complexity that make **sim-to-real techniques** crucial for success. Training robots purely in the real world is often slow, risky, and expensive – simulation offers a faster and safer alternative. However, classical simulators have struggled with the **“reality gap”**: the visual and physical discrepancies between simulated environments and the real world. In the past, researchers...
    1 hr 3 min
  • Episode 31: WMPO: Training Robot Policies in Imagination
    ## Introduction – From Imitation to Imagination
    Imagine a household robot that can **follow human instructions** flawlessly when everything goes as in training, but gets hopelessly stuck when something unexpected happens. This is the limitation of today’s **Vision-Language-Action (VLA)** models: they learn from **expert demonstrations** (imitation learning) and can repeat those behaviors, but they **can’t learn from their own mistakes**. If a task deviates from the scripted training scenar...
    56 min
  • Episode 30: Learning a Thousand Tasks: Oxford’s MT3 and the Quest for Generalist Robots
    ## Introduction
    Robots that can learn **many diverse tasks** efficiently have long been a goal in robotics. Humans can watch a single demonstration and immediately attempt a new task, but robots typically require _massive training data_ for each new skill. Recent research is closing this gap with innovative multi-task learning approaches. The Robot Learning Lab at Oxford (Imperial College London) recently demonstrated a system that learned **1,000 distinct manipulation tasks in a single da...
    1 hr 7 min
  • Episode 29: BFM-Zero: Unsupervised RL for Humanoid Control with a Shared Latent Skill Space
    ## Introduction
    Building generalist robot controllers that can perform many tasks on demand is a major goal in vision-language-action (VLA) models and robotics. **BFM-Zero** (Behavioral Foundation Model – Zero) is a recent framework tackling this by training a **single humanoid policy** that can be _prompted_ to execute diverse skills without retraining. Unlike traditional approaches that rely on carefully crafted task rewards or behavior cloning from demonstrations, BFM-Zero uses **unsupe...
    45 min
  • Episode 28: Self Forcing: Bridging the Train–Test Gap in Autoregressive Video Diffusion
    Recent advances in text-to-video generation have achieved impressive fidelity and complex temporal dynamics in short clips. However, many state-of-the-art video diffusion models operate in a **non-causal** fashion: they generate an entire video in one go with bidirectional attention across time. This means future frames can influence past ones during generation, yielding high quality but precluding real-time use cases where future information isn’t available at inference time. By contrast, **...
    1 hr 8 min
  • Episode 27: Smol-Training Playbook: Technical Review of Tools and Frameworks
    ## Introduction
    The **Smol-Training Playbook** from Hugging Face is a comprehensive guide that distills best practices for training large language models (LLMs) on a relatively small compute budget. Central to this playbook are several specialized frameworks and libraries that make efficient training feasible. In this report, we examine the key tools imported or referenced in the playbook’s training pipeline – including **Nanotron**, **FlashAttention**, **Liger**, **DataTrove**, **TRL**, a...
    1 hr 12 min
  • Episode 25: GEN‑0: Pushing the Frontiers of Embodied AI with Harmonic Reasoning
    Modern embodied AI models are rapidly evolving, and _GEN‑0_ by Generalist AI represents a significant leap. Announced in late 2025, GEN‑0 is an embodied foundation model built to _think_ and _act_ in the real world at unprecedented scale. In this article, we dive into what sets GEN‑0 apart from prior vision-language-action (VLA) systems – from its novel **Harmonic Reasoning** architecture to its massive real-world dataset – and compare it with recent models like VLA-0, RT-2, PaLM-E, π₀ (pi-ze...
    49 min
  • Episode 26: SoftMimic: Teaching Humanoids to Move Softly
    ## The Problem with Rigid Controllers
    Most robots trained to imitate human motions have a hidden flaw: they’re **too stiff**. When an imitation-trained humanoid encounters unexpected contact – say it brushes against a table, misjudges an object’s position, or bumps into a person – it treats that deviation as an error and tries to correct it _aggressively_. The result is often _large, uncontrolled forces_ against the obstacle, making the robot’s behavior brittle and potentially dangerous. In ...
    34 min

About Embodied AI 101

From the publisher's feed

Podcast about embodied AI and robotics