**Meta’s SAM‑3D** is a new **foundation model for 3D** vision that extends the 2D **Segment Anything Model (SAM)** into the third dimension. It actually consists of two specialized models – **SAM‑3D Objects** (for general 3D object and scene reconstruction) and **SAM‑3D Body** (for full 3D human pose and shape estimation). Both were trained on unprecedented datasets using novel training pipelines. In this post, we’ll dive into **how SAM‑3D was trained** – covering the datasets (real vs synthe...

## Motivation: Why Sim-to-Real Matters Now

Robotic manipulation and navigation systems are reaching levels of complexity that make **sim-to-real techniques** crucial for success. Training robots purely in the real world is often slow, risky, and expensive – simulation offers a faster and safer alternative. However, classical simulators have struggled with the **“reality gap”**: the visual and physical discrepancies between simulated environments and the real world. In the past, researchers...

## Introduction – From Imitation to Imagination

Imagine a household robot that can **follow human instructions** flawlessly when everything goes as in training, but gets hopelessly stuck when something unexpected happens. This is the limitation of today’s **Vision-Language-Action (VLA)** models: they learn from **expert demonstrations** (imitation learning) and can repeat those behaviors, but they **can’t learn from their own mistakes**. If a task deviates from the scripted training scenar...

## Introduction

Robots that can learn **many diverse tasks** efficiently have long been a goal in robotics. Humans can watch a single demonstration and immediately attempt a new task, but robots typically require _massive training data_ for each new skill. Recent research is closing this gap with innovative multi-task learning approaches. The Robot Learning Lab at Oxford (Imperial College London) recently demonstrated a system that learned **1,000 distinct manipulation tasks in a single da...

## Introduction

Building generalist robot controllers that can perform many tasks on demand is a major goal in vision-language-action (VLA) models and robotics. **BFM-Zero** (Behavioral Foundation Model – Zero) is a recent framework tackling this by training a **single humanoid policy** that can be _prompted_ to execute diverse skills without retraining. Unlike traditional approaches that rely on carefully crafted task rewards or behavior cloning from demonstrations, BFM-Zero uses **unsupe...

Recent advances in text-to-video generation have achieved impressive fidelity and complex temporal dynamics in short clips. However, many state-of-the-art video diffusion models operate in a **non-causal** fashion: they generate an entire video in one go with bidirectional attention across time. This means future frames can influence past ones during generation, yielding high quality but precluding real-time use cases where future information isn’t available at inference time. By contrast, **...

## Introduction

The **Smol-Training Playbook** from Hugging Face is a comprehensive guide that distills best practices for training large language models (LLMs) on a relatively small compute budget. Central to this playbook are several specialized frameworks and libraries that make efficient training feasible. In this report, we examine the key tools imported or referenced in the playbook’s training pipeline – including **Nanotron**, **FlashAttention**, **Liger**, **DataTrove**, **TRL**, a...

## The Problem with Rigid Controllers

Most robots trained to imitate human motions have a hidden flaw: they’re **too stiff**. When an imitation-trained humanoid encounters unexpected contact – say it brushes against a table, misjudges an object’s position, or bumps into a person – it treats that deviation as an error and tries to correct it _aggressively_. The result is often _large, uncontrolled forces_ against the obstacle, making the robot’s behavior brittle and potentially dangerous. In ...

Modern embodied AI models are rapidly evolving, and _GEN‑0_ by Generalist AI represents a significant leap. Announced in late 2025, GEN‑0 is an embodied foundation model built to _think_ and _act_ in the real world at unprecedented scale. In this article, we dive into what sets GEN‑0 apart from prior vision-language-action (VLA) systems – from its novel **Harmonic Reasoning** architecture to its massive real-world dataset – and compare it with recent models like VLA-0, RT-2, PaLM-E, π₀ (pi-ze...