Lance: https://www.linkedin.com/in/lance-martin-64a33b5/

How Context Fails: https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html
How New Buzzwords Get Created: https://www.dbreunig.com/2025/07/24/why-the-term-context-engineering-matters.html
Content Engineering: https://x.com/RLanceMartin/status/1948441848978309358 https://rlancemartin.github.io/2025/06/23/context_engineering/ https://docs.google.com/presentation/d/16aaXLu40GugY-kOpqDU4e-S0hD1FmHcNyF0rRRnb1OU/edit?usp=sharing
Manus Post: https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
Cognition Post: https://cognition.ai/blog/dont-build-multi-agents
Multi-Agent Researcher: https://www.anthropic.com/engineering/multi-agent-research-system
Human-in-the-loop + Memory: https://github.com/langchain-ai/agents-from-scratch
- Bitter Lesson in AI Engineering -
Hyung Won Chung on the Bitter Lesson in AI Research: https://www.youtube.com/watch?v=orDKvo8h71o
Bitter Lesson w/ Claude Code: https://www.youtube.com/watch?v=Lue8K2jqfKk&t=1s
Learning the Bitter Lesson in AI Engineering: https://rlancemartin.github.io/2025/07/30/bitter_lesson/
Open Deep Research: https://github.com/langchain-ai/open_deep_research https://academy.langchain.com/courses/deep-research-with-langgraph
Scaling and building things that "don't yet work": https://www.youtube.com/watch?v=p8Jx4qvDoSo
- Frameworks -
Roast framework at Shopify / standardization of orchestration tools: https://www.youtube.com/watch?v=0NHCyq8bBcM
MCP adoption within Anthropic / standardization of protocols: https://www.youtube.com/watch?v=xlEQ6Y3WNNI
How to think about frameworks: https://blog.langchain.com/how-to-think-about-agent-frameworks/
RAG benchmarking: https://rlancemartin.github.io/2025/04/03/vibe-code/
Simon's talk with memory-gone-wrong: https://simonwillison.net/2025/Jun/6/six-months-in-llms/

Today we are joined by Gorkem and Batuhan from Fal.ai, the fastest growing generative media inference provider. They recently raised a $125M Series C and crossed $100M ARR. We covered how they pivoted from dbt pipelines to diffusion models inference, what were the models that really changed the trajectory of image generation, and the future of AI videos. Enjoy!

00:00 - Introductions
04:58 - History of Major AI Models and Their Impact on Fal.ai
07:06 - Pivoting to Generative Media and Strategic Business Decisions
10:46 - Technical discussion on CUDA optimization and kernel development
12:42 - Inference Engine Architecture and Kernel Reusability
14:59 - Performance Gains and Latency Trade-offs
15:50 - Discussion of model latency importance and performance optimization
17:56 - Importance of Latency and User Engagement
18:46 - Impact of Open Source Model Releases and Competitive Advantage
19:00 - Partnerships with closed source model developers
20:06 - Collaborations with Closed-Source Model Providers
21:28 - Serving Audio Models and Infrastructure Scalability
22:29 - Serverless GPU infrastructure and technical stack
23:52 - GPU Prioritization: H100s and Blackwell Optimization
25:00 - Discussion on ASICs vs. General Purpose GPUs
26:10 - Architectural Trends: MMDiTs and Model Innovation
27:35 - Rise and Decline of Distillation and Consistency Models
28:15 - Draft Mode and Streaming in Image Generation Workflows
29:46 - Generative Video Models and the Role of Latency
30:14 - Auto-Regressive Image Models and Industry Reactions

31:35 - Discussion of OpenAI's Sora and competition in video generation
34:44 - World Models and Creative Applications in Games and Movies
35:27 - Video Models’ Revenue Share and Open-Source Contributions
36:40 - Rise of Chinese Labs and Partnerships
38:03 - Top Trending Models on Hugging Face and ByteDance's Role
39:29 - Monetization Strategies for Open Models
40:48 - Usage Distribution and Model Turnover on FAL
42:11 - Revenue Share vs. Open Model Usage Optimization
42:47 - Moderation and NSFW Content on the Platform
44:03 - Advertising as a key use case for generative media
45:37 - Generative Video in Startup Marketing and Virality
46:56 - LoRA Usage and Fine-Tuning Popularity
47:17 - LoRA ecosystem and fine-tuning discussion
49:25 - Post-Training of Video Models and Future of Fine-Tuning
50:21 - ComfyUI Pipelines and Workflow Complexity
52:31 - Requests for startups and future opportunities in the space
53:33 - Data Collection and RedPajama-Style Initiatives for Media Models
53:46 - RL for Image and Video Models: Unknown Potential
55:11 - Requests for Models: Editing and Conversational Video Models
57:12 - VO3 Capabilities: Lip Sync, TTS, and Timing
58:23 - Bitter Lesson and the Future of Model Workflows
58:44 - FAL's hiring approach and team structure
59:29 - Team Structure and Scaling Applied ML and Performance Teams
1:01:41 - Developer Experience Tools and Low-Code/No-Code Integration
1:03:04 - Improving Hiring Process with Public Challenges and Benchmarks
1:04:02 - Closing Remarks and Culture at FAL

Our chat with Ari shows that data curation is the most impactful and underinvested area in AI. He argues that the prevailing focus on model architecture and compute scaling overlooks the "bitter lesson" that "models are what they eat." Effective data curation—a sophisticated process involving filtering, rebalancing, sequencing (curriculum), and synthetic data generation—allows for training models that are simultaneously faster, better, and smaller. Morcos recounts his personal journey from focusing on model-centric inductive biases to realizing that data quality is the primary lever for breaking the diminishing returns of naive scaling laws. Datology's mission is to automate this complex curation process, making state-of-the-art data accessible to any organization and enabling a new paradigm of AI development where data efficiency, not just raw scale, drives progress.

Timestamps

00:00 Introduction

00:46 What is Datology? The mission to train models faster, better, and smaller through data curation.

01:59 Ari's background: From neuroscience to realizing the "Bitter Lesson" of AI.

05:30 Key Insight: Inductive biases from architecture become less important and even harmful as data scale increases.

08:08 Thesis: Data is the most underinvested area of AI research relative to its impact.

10:15 Why data work is culturally undervalued in research and industry.

12:19 How self-supervised learning changed everything, moving from a data-scarce to a data-abundant regime.

17:05 Why automated curation is superior to human-in-the-loop, citing the DCLM study.

19:22 The "Elephants vs. Dogs" analogy for managing data redundancy and complexity.

22:46 A brief history and commentary on key datasets (Common Crawl, GitHub, Books3).

26:24 Breaking naive scaling laws by improving data quality to maintain high marginal information gain.

29:07 Datology's demonstrated impact: Achieving baseline performance 12x faster.

34:19 The business of data: Datology's moat and its relationship with open-source datasets.

39:12 Synthetic Data Explain

ed: The difference between risky "net-new" creation and powerful "rephrasing."

49:02 The Resurgence of Curriculum Learning: Why ordering data matters in the underfitting regime.

52:55 The Future of Training: Optimizing pre-training data to make post-training more effective.

54:49 Who is training their own models and why (Sovereign AI, large enterprises).

57:24 "Train Smaller": Why inference cost makes smaller, specialized models the ultimate goal for enterprises.

01:00:19 The problem with model pruning and why data-side solutions are complementary.

01:03:03 On finding the smallest possible model for a given capability.

01:06:49 Key learnings from the RC foundation model collaboration, proving that data curation "stacks."

01:09:46 Lightning Round: What data everyone wants & who should work at Datology.

01:14:24 Commentary on Meta's superintelligence efforts and Yann LeCun's role.

Jeff Huber of Chroma joins us to talk about what actually matters in vector databases in 2025, why “modern search for AI” is different, and how to ship systems that don’t rot as context grows.

Full show notes: https://www.latent.space/p/chroma

00:00 Introductions
00:48 Why Build Chroma
02:55 Information Retrieval vs. Search
04:29 Staying Focused in a Competitive AI Market
08:08 Building Chroma Cloud
12:15 Context Engineering and the Problems with RAG
16:11 Context Rot
21:49 Prioritizing Context Quality
27:02 Code Indexing and Retrieval Strategies
32:04 Chunk Rewriting and Query Optimization for Code
34:07 Transformer Architecture Evolution and Retrieval Systems
38:06 Memory as a Benefit of Context Engineering
40:13 Structuring AI Memory and Offline Compaction
45:46 Lessons from Previous Startups and Building with Purpose
47:32 Religion and Values in Silicon Valley
50:18 Company Culture, Design, and Brand Consistency
52:36 Hiring at Chroma: Designers, Researchers, and Engineers

Greg Brockman, co-founder and president of OpenAI, joins us to talk about GPT-5 and GPT-OSS, the future of software engineering, why reinforcement learning is still scaling, and how OpenAI is planning to get to AGI.

00:00 Introductions
01:04 The Evolution of Reasoning at OpenAI
04:01 Online vs Offline Learning in Language Models
06:44 Sample Efficiency and Human Curation in Reinforcement Learning
08:16 Scaling Compute and Supercritical Learning
13:21 Wall clock time limitations in RL and real-world interactions
16:34 Experience with ARC Institute and DNA neural networks
19:33 Defining the GPT-5 Era
22:46 Evaluating Model Intelligence and Task Difficulty
25:06 Practical Advice for Developers Using GPT-5
31:48 Model Specs
37:21 Challenges in RL Preferences (e.g., try/catch)
39:13 Model Routing and Hybrid Architectures in GPT-5
43:58 GPT-5 pricing and compute efficiency improvements
46:04 Self-Improving Coding Agents and Tool Usage
49:11 On-Device Models and Local vs Remote Agent Systems
51:34 Engineering at OpenAI and Leveraging LLMs
54:16 Structuring Codebases and Teams for AI Optimization
55:27 The Value of Engineers in the Age of AGI
58:42 Current state of AI research and lab diversity
01:01:11 OpenAI’s Prioritization and Focus Areas
01:03:05 Advice for Founders: It's Not Too Late
01:04:20 Future outlook and closing thoughts
01:04:33 Time Capsule to 2045: Future of Compute and Abundance
01:07:07 Time Capsule to 2005: More Problems Will Emerge

• 00:00:00 Welcome and Guest Introduction
• 00:01:18 Tulu, OVR, and the RLVR Journey
• 00:03:40 Industry Approaches to Post-Training and Preference Data
• 00:06:08 Understanding RLVR and Its Impact
• 00:06:18 Agents, Tool Use, and Training Environments
• 00:10:34 Open Data, Human Feedback, and Benchmarking
• 00:12:44 Chatbot Arena, Sycophancy, and Evaluation Platforms
• 00:15:42 RLHF vs RLVR: Books, Algorithms, and Future Directions
• 00:17:54 Frontier Models: Reasoning, Hybrid Models, and Data
• 00:22:11 Search, Retrieval, and Emerging Model Capabilities
• 00:29:23 Tool Use, Curriculum, and Model Training Challenges
• 00:38:06 Skills, Planning, and Abstraction in Agent Models
• 00:46:50 Parallelism, Verifiers, and Scaling Approaches
• 00:54:33 Overoptimization and Reward Design in RL
• 01:02:27 Open Models, Personalization, and the Model Spec
• 01:06:50 Open Model Ecosystem and Infrastructure
• 01:13:05 Meta, Hardware, and the Future of AI Competition
• 01:15:42 Building an Open DeepSeek and Closing Thoughts

We first had Nathan on to give us his RLHF deep dive when he was joining AI2, and now he’s back to help us catch up on the evolution to RLVR (Reinforcement Learning with Verifiable Rewards), first proposed in his Tulu 3 paper. While RLHF remains foundational, RLVR has emerged as a powerful approach for training models on tasks with clear success criteria and using verifiable, objective functions as reward signals—particularly useful in domains like math, code correctness, and instruction-following. Instead of relying solely on subjective human feedback, RLVR leverages deterministic signals to guide optimization, making it more scalable and potentially more reliable across many domains. However, he notes that RLVR is still rapidly evolving, especially regarding how it handles tool use and multi-step reasoning.

We also discussed the Tulu model series, a family of instruction-tuned open models developed at AI2. Tulu is designed to be a reproducible, state-of-the-art post-training recipe for the open community. Unlike frontier labs like OpenAI or Anthropic, which rely on vast and often proprietary datasets, Tulu aims to distill and democratize best practices for instruction and preference tuning. We are impressed with how small eval suites, careful task selection, and transparent methodology can rival even the best proprietary models on specific benchmarks.

One of the most fascinating threads is the challenge of incorporating tool use into RL frameworks. Lambert highlights that while you can prompt a model to use tools like search or code execution, getting the model to reliably learn when and how to use them through RL is much harder. This is compounded by the difficulty of designing reward functions that avoid overoptimization—where models learn to “game” the reward signal rather than solve the underlying task. This is particularly problematic in code generation, where models might reward hack unit tests by inserting pass statements instead of correct logic. As models become more agentic and are expected to plan, retrieve, and act across multiple tools, reward design becomes a critical bottleneck.

Other topics covered:

- The evolution from RLHF (Reinforcement Learning from Human Feedback) to RLVR (Reinforcement Learning from Verifiable Rewards)
- The goals and technical architecture of the Tulu models, including the motivation to open-source post-training recipes
- Challenges of tool use in RL: verifiability, reward design, and scaling across domains
- Evaluation frameworks and the role of platforms like Chatbot Arena and emerging “arena”-style benchmarks
- The strategic tension between hybrid reasoning models and unified reasoning models at the frontier
- Planning, abstraction, and calibration in reasoning agents and why these concepts matter
- The future of open-source AI models, including DeepSeek, OLMo, and the potential for an “American DeepSeek”
- The importance of model personality, character tuning, and the model spec paradigm
- Overoptimization in RL settings and how it manifests in different domains (control tasks, code, math)
- Industry trends in inference-time scaling and model parallelism

Finally, the episode closes with a vision for the future of open-source AI. Nathan has now written up his ambition to build an “American DeepSeek”—a fully open, end-to-end reasoning-capable model with transparent training data, tools, and infrastructure. He emphasizes that open-source AI is not just about weights; it’s about releasing recipes, evaluations, and methods that lower the barrier for everyone to build and understand cutting-edge systems. It would seem the

This is a recap episode that ends with a short fresh interview on the future of Windsurf + Cognition with Jeff Wang and Scott Wu at the end.

As the story of Windsurf as an independent company has come to a dramatic close with Google and Cognition, we’re taking this opportunity to look back at our coverage of Windsurf over the last 3 years. Here’s a brief timeline with related links.

• Jun 2021 - Exafunction founded

• Oct 2022 - Codeium pivot https://windsurf.com/blog/beta-launch-announcement

• Dec 2022 - “Copilot for X” https://www.latent.space/p/what-building-copilot-for-x-really

• Mar 2023 - Codeium first episode, LS episode 2 https://www.latent.space/p/varun-mohan

• July 2023 - “How to Make AI UX Your Moat" ****https://www.latent.space/p/ai-ux-moat

• Mar 2024 - Cognition Devin launch https://www.youtube.com/watch?v=fjHtjT7GO1c

• Jun 2024 - Scott @ AI Engineer https://www.youtube.com/watch?v=T7NWjoD_OuY

• Jun 2024 - Kevin @ AI Engineer https://www.youtube.com/watch?v=DuZXbinJ4Uc

• Nov 2024 - “Enterprise Infra Native” https://www.latent.space/p/enterprise

• Nov 2024 - Windsurf launch, LS Episode https://www.latent.space/p/windsurf

• Mar 2025 - Kevin Hou @ AI Engineer https://www.youtube.com/watch?v=bVNNvWq6dKo

• Jun 2025 - Scott @ AI Engineer https://www.youtube.com/watch?v=MI83buT_23o

• Jun 2025 - Kevin Hou @ AI Engineer https://www.youtube.com/watch?v=JVuNPL5QO8Q

• Jul 2025 - Jeff + Scott, CogSurf Episode ← new one, released here.

We hope this serves as food for thought for students of history, and a reintroduction to the Latent Space extended universe and backlog, for those of you who are new. Welcome!

Timestamps

[00:02:07] Mar 2024 Codeium @ LS
[00:52:36] Mar 2024 Devin Launch Video
[00:54:28] Jun 2024 Codeium @ AIE SF
[01:12:14] Jun 2024 Cognition @ AIE SF
[01:30:53] Nov 2024 Windsurf Launch Video
[01:37:16] Nov 2024 Windsurf Launch @ LS
[02:43:10] Feb 2025 Windsurf @ AIE NYC
[03:03:27] Jun 2025 Cognition @ AIE SF
[03:18:50] June 2025 Windsurf @ AIE SF
[03:34:23] July 2025 - Cognition + Windsurf

• 00:00:00 Mar 2024 Codeium @ LS
• 00:52:36 Mar 2024 Devin Launch Video
• 00:54:28 Jun 2024 Codeium @ AIE SF
• 01:12:14 Jun 2024 Cognition @ AIE SF
• 01:30:53 Nov 2024 Windsurf Launch Video
• 01:37:16 Nov 2024 Windsurf Launch @ LS
• 02:43:10 Feb 2025 Windsurf @ AIE NYC
• 03:03:27 Jun 2025 Cognition @ AIE SF
• 03:18:50 June 2025 Windsurf @ AIE SF
• 03:34:23 July 2025 - Cognition + Windsurf

ChatGPT handles 2.5B prompts/day and is on track to match Google's daily searches by end of 2026. AI agents don't browse like us—they crave queryable, chunkable data for tools like ChatGPT & Perplexity. A new industry is being born, some are calling it AI SEO, others GEO, but what is clear is that it drives amazing results. Businesses are seeing 2-4x higher conversion from visitors coming from AI compared to traditional search. Robert McCloy is the co-founder of Scrunch AI (https://scrunchai.com/), a fast growing company that helps brands and businesses re-write their content on the fly based on what agents are looking for.

Akshay Agrawal joins us to talk about Marimo and their vision for the future of Python notebooks, and how it’s the perfect canvas for AI-driven data analysis.

0:00 Introduction
0:46 Overview of Marimo and Its Features
2:33 Origin Story and Motivation Behind Marimo
4:26 Demo: Classical Machine Learning with MNIST in Marimo
6:52 Notebook Compatibility and Conversion from Jupyter
7:42 Demo: Interactive Notebook with Custom UI and Layout
10:08 AI-Native Utilities and Code Generation with Language Models
11:36 Dependency Management and Integration with UV Package Manager
13:00 Demo: Data Annotation Workflow Using a PS5 Controller
15:51 Starting from Scratch: Blank Canvas AI Use Cases
18:27 Context Formatting for AI Code Generation
19:54 Chat Interface and Local/Remote Model Support
21:01 WebAssembly Support and MoLab Cloud-Hosted Notebooks
23:21 Future Plans and Breaking Out of Old Notebook Habits
25:40 Running Marimo Notebooks as Scripts or Data Apps
26:44 Exploring AI Agents and Community Contributions
26:56 Call to Action: How to Get Started and Contribute

Saoud Rizwan and Pash from Cline joined us to talk about why fast apply models got bitter lesson'd, how they pioneered the plan + act paradigm for coding, and why non-technical people use IDEs to do marketing and generate slides.

Full writeup: https://www.latent.space/p/cline

X: https://x.com/latentspacepod

Chapters:

00:00 - Introductions
01:35 - Plan and Act Paradigm
05:37 - Model Evaluation and Early Development of Cline
08:14 - Use Cases of Cline Beyond Coding
09:09 - Why Cline is a VS Code Extension and Not a Fork
12:07 - Economic Value of Programming Agents
16:07 - Early Adoption for MCPs
19:35 - Local vs Remote MCP Servers
22:10 - Anthropic's Role in MCP Registry
22:49 - Most Popular MCPs and Their Use Cases
25:26 - Challenges and Future of MCP Monetization
27:32 - Security and Trust Issues with MCPs
28:56 - Alternative History Without MCP
29:43 - Market Positioning of Coding Agents and IDE Integration Matrix
32:57 - Visibility and Autonomy in Coding Agents
35:21 - Evolving Definition of Complexity in Programming Tasks
38:16 - Forks of Cline and Open Source Regrets
40:07 - Simplicity vs Complexity in Agent Design
46:33 - How Fast Apply Got Bitter Lesson'd
49:12 - Cline's Business Model and Bring-Your-Own-API-Key Approach
54:18 - Integration with OpenRouter and Enterprise Infrastructure
55:32 - Impact of Declining Model Costs
57:48 - Background Agents and Multi-Agent Systems
1:00:42 - Vision and Multi-Modalities
1:01:07 - State of Context Engineering
1:07:37 - Memory Systems in Coding Agents
1:10:14 - Standardizing Rules Files Across Agent Tools
1:11:16 - Cline's Personality and Anthropomorphization
1:12:55 - Hiring at Cline and Team Culture