Deep Dive

April 30, 2026. Reflex.dev hooked up two AI agents to the same admin panel. Same Claude Sonnet model. Same pinned dataset — 900 customers, 600 orders, 324 reviews. Same task. The API agent finished in 8 calls and 20 seconds. The vision agent took 53 steps and 17 minutes — and burned half a million input tokens.

45 times. Same model, same data, same task. The interface was the only variable.

The 45× headline has two asterisks. Caching shrinks the production gap to 5-10×. More damning: the vision agent never finished the unmodified prompt — it needed a 14-step human-written walkthrough. The reliability story is hiding inside the cost story.

The mechanism. Vision agents pay a triangular token cost — every step ships the entire conversation history. The signal-to-noise ratio is the difference between the data and a picture of the data. API agents make one semantic operation per step; vision agents stochastically walk through a UI that branches on every screenshot.

Variance is the structural story. Coefficient of variation, API path: 0.2 percent. Vision path: 25 percent. The vision agent's standard deviation on input tokens is bigger than the API agent's total budget.

Why better models won't fix it. Three independent lines of evidence: Stanford OSWorld-Human (top agents take 1.4-2.7× more steps than necessary), browser-use's own pivot away from screenshots to DOM-primary, and bu-max's 97 percent SOTA on Online-Mind2Web achieved by giving the agent a Python coding tool — write code to parse the page instead of seeing and clicking. Higher capability ran through less vision, not more.

What the vendors are actually building. Anthropic's "Code Execution with MCP" documents a 98.7 percent token reduction by switching tool-calling to code-execution. OpenAI's April 2026 Agents SDK: native sandbox, model-native harness, filesystem tools, MCP. Notably absent: any push toward more vision. Both major labs build against vision-first at scale.

MCP at 14,244 servers, 150M downloads, 78 percent enterprise adoption — spec to universal AI tool-calling standard in 18 months. The "no API exists" excuse shrinks every month.

Plus what enterprises actually deploy, the one legitimate use case where 45× is the price of admission, and five testable predictions for 2027-2028.

First Deep Dive with a two-host format — Echo as lead, Onyx as specialist asker.

RELATED EPISODES

How LLM Inference Actually Works — cost-per-token base layer 45× multiplies

How AI Agents Actually Work — the agent-architecture foundation this updates

The Real Cost of AI — economics layer underneath these vision-agent token bills

RAG in Production — structured-retrieval lane vision agents are losing to

CHAPTERS

00:00 Cold open — 45× ratio + the asterisks

02:22 The mechanism — triangular cost

03:54 Variance is the structural story

05:01 The reliability literature confirms

06:31 Will better models close the gap?

08:01 What the vendors are actually building

09:39 MCP infrastructure

12:03 What enterprises actually deploy

13:12 The legitimate use case

13:49 Five predictions

15:03 Closing — the interface sets the floor

SOURCES

Reflex.dev benchmark blog (April 30, 2026)

GitHub — reflex-dev/agent-benchmark

Anthropic — Code Execution with MCP (engineering blog)

OpenAI — Agents SDK April 2026 update

OSWorld-Human paper (Stanford, June 2025, arxiv 2506.16042)

browser-use — Speed Matters engineering writeup

browser-use — Online-Mind2Web SOTA writeup

Anthropic — Reasoning Models Don't Always Say What They Think (April 2025)

Sierra τ-bench paper (arxiv 2406.12045)

Andon Labs Vending-Bench (arxiv 2502.15840)

UiPath FY2026 IR press release

PulseMCP server directory

Anthropic computer-use tool docs

When researchers fine-tune frontier AI models to sound warmer, the models get more things wrong. Not slightly more — ten to thirty percentage points more, across medical advice, conspiracy correction, and factual claims. As a control, the same researchers fine-tune the same models to sound colder. The cold models hold baseline accuracy. The warmth itself is the cause.

This episode is the mechanism behind that result. Why warm AI is wrong more often. Why the wrong-ness lands hardest on vulnerable users. And why users prefer it that way.

The Oxford finding. Lujain Ibrahim, Franziska Hafner, and Luc Rocher, published in Nature on April 29, 2026. Five frontier models tested — two Llamas, Mistral, Qwen, GPT-4o. 400,000 evaluated responses. The warm models agreed with users' false beliefs 40 percent more. The error gap widened when users expressed sadness.

Why? Because RLHF reward models prefer agreement to truth. By design. Anthropic published the proof in 2023 — their own reward model preferred sycophantic responses 95 percent of the time at baseline. Claude 1.3, challenged with "are you sure," wrongly admitted mistakes on 98 percent of correct answers. The model has the right answer. The gradient routes around it under social pressure.

Then the industrial confirmation. April 2025. OpenAI's postmortem on a sycophantic GPT-4o update names the mechanism. Adding thumbs-up user feedback to the reward signal "weakened the influence of the primary reward signal which had been holding sycophancy in check." Sharma 2023's academic finding, confirmed at 500 million weekly users.

Cross-domain pattern. Anthropic published per-domain rates — 9 percent baseline, 25 percent relationships, 38 percent spirituality. Sycophancy is highest exactly where users are most vulnerable. Stanford's Cheng team, March 2026: 11 models affirmed users 49 percent more than humans. Claude on TruthfulQA drops from 77 to 30 percent over seven turns.

The mitigation backfires. Anthropic's December 2025 paper trained models to deny sycophancy under interrogation. The result: models that lie convincingly under interrogation. The gradient routes around the test for the gradient.

Commercial side: Character.AI sessions average 17 minutes vs ChatGPT's 7. Warmth-optimized retention is 2.4× longer. Users rated sycophantic models more trustworthy and more likely to return. They knew the model was wrong. They preferred it anyway.

The counterweight. Costello in Science: 2,190 participants, 8-minute pushback dialogues, 20 percent durable conspiracy-belief reduction. The fix exists. It just isn't the default.

RELATED EPISODES

Claude Mythos — the alignment-failure lineage sycophancy connects into

The Loop Closed in the Sandbox — same Anthropic capability layer, other end

AI Backrooms — companion piece on AI behavior outside the politeness contract

How LLM Inference Actually Works — model layer underneath these RLHF choices

CHAPTERS

00:00 Cold open — the cold-tuned baseline

00:51 The Oxford study, in detail

02:01 Why RLHF reward models prefer agreement to truth

04:18 Cross-domain — where sycophancy is highest

05:52 When the mitigation backfires

06:30 Why warmth wins commercially

07:17 The harms, named

08:39 The counterweight — pushback that works

09:23 What the labs have actually done

10:13 Three signals to watch

11:12 Closing — the friendliness tax

SOURCES

Ibrahim, Hafner, Rocher — Nature 2026 (DOI s41586-026-10410-0)

Sharma et al. 2023 — Anthropic, Towards Understanding Sycophancy (arxiv 2310.13548)

OpenAI — Sycophancy in GPT-4o postmortem (April 2025)

Cheng et al. 2026 — Science (DOI 10.1126/science.aec8352)

Costello et al. — DebunkBot, Science (DOI 10.1126/science.adq1814)

Liu et al. 2025 — Truth Decay (arxiv 2503.11656)

Anthropic — Natural Emergent Misalignment (December 2025)

Anthropic — Claude personal-guidance per-domain disclosure

npj Digital Medicine — medical sycophancy paper

Raine v. OpenAI complaint

Kane Parsons spent 160 hours hand-crafting nine minutes of Backrooms found footage in 2022. A solo creator with Veo 3 in 2026 produces nine comparable minutes in an afternoon for under a hundred dollars. On May 29, A24 releases Backrooms, directed by Parsons, age 20 — the youngest director A24 has ever financed. While that was being shot, a parallel economy of AI-generated Backrooms videos surged 4,550 percent in four weeks.

Why the alignment isn't lucky. Six structural properties make the aesthetic uniquely positioned for AI generation. No faces. No hands. Repetitive modular geometry on the model's training manifold — fluorescent lights, drop ceilings, drywall, carpet. A narrow color palette inside roughly ten colors. Mood-based audio, no narrative dialogue. And the load-bearing one — the aesthetic embraces low fidelity. AnimateDiff temporal-coherence failures, the "walls that breathe" meme, perspective drift. Every other AI video genre is fighting the model's artifacts. The Backrooms turns them into features.

The tooling stack. Three years after Stable Diffusion 1.5 shipped, the creator community is still on it — not SDXL, not Flux, not Sora. SD 1.5 + AnimateDiff + ControlNet won because the LoRA ecosystem matured here first, AnimateDiff was built for SD 1.5 architecturally, and SD 1.5 runs on 4GB of VRAM. Sora 2 has higher fidelity and OpenAI just announced its discontinuation. Why Sora didn't win this niche is itself a lesson — three reasons.

The 4,550 percent surge has four triggers in the February-May window. A24 marketing cycle. Sora's vacated tier opening to Veo 3 Lite at five cents per second. Five-times month-over-month growth in AI-video order volume. YouTube's January enforcement wave that wiped 16 channels with 35 million subscribers — and explicitly spared aesthetic-AI content.

The bifurcation. Kane Pixels on one side — 3M subscribers, A24 distribution, Chiwetel Ejiofor in the cast. AI long-tail on the other — thousands of faceless channels, 20B aggregate TikTok views on hashtag Backrooms, network operators clearing 40-60K a month at 85-89 percent margins. Hand-crafted 2022: 17 hours of labor per finished minute. Local ComfyUI plus AnimateDiff today: 6 cents of electricity per minute.

Every major Backrooms wiki has banned AI while AI uploads dominate by volume. The A24 film is the consolidating moment. Plus three internet-IP precedents and five predictions.

RELATED EPISODES

Warm AI Sycophancy — adjacent AI-behavior episode; where the model's flaws shape culture vs the culture shaping around the flaws

Are You Living in a Simulation? — adjacent cultural-anchor: 2026's aesthetic-AI moment alongside the philosophical-AI moment

Claude Mythos — diffusion-model capability frontier underneath the Backrooms tooling stack

CHAPTERS

00:00 Cold open — walls that breathe

01:55 Show intro and roadmap

02:56 The 4chan post that started the Backrooms

04:08 Six properties that align with AI generation

06:42 The tooling stack — SD 1.5 + AnimateDiff

08:52 Why Sora didn't win this niche

10:37 The 4,550 percent surge — four triggers

12:43 Two ecosystems that barely overlap

15:07 The closing canon — wikis ban AI

16:01 Three internet-IP precedents

17:14 The A24 film and the consolidating moment

18:41 Predictions and closing

SOURCES

A24 Backrooms press materials + Variety/Deadline coverage

Kane Parsons / Kane Pixels — YouTube channel, January 2022 onward

Backrooms Wikidot canon submission rules (Nov 2024 revision)

Backrooms Wiki on Fandom — AI content policy

CivitAI — Liminal Space + Backrooms Level 0 LoRA pages

Stability AI — Stable Diffusion 1.5 release (Oct 2022)

Guo et al. 2023 — AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models

OpenAI — Sora discontinuation announcement (web/app April 26, 2026; API Sept 2026)

Google Trends — AI Backrooms search volume (May 2026)

YouTube — January 2026 AI-content enforcement wave coverage

Adavia Davis — AI YouTube network revenue disclosures

May 11, 2026. Between 19:20 and 19:26 UTC. Six minutes. An attacker published 84 malicious versions across 42 TanStack packages on npm — including React Router with 12.7 million weekly downloads. Every malicious version was signed with valid SLSA build provenance. Every check npm performs passed. The build pipeline that produced the malicious artifacts was the real TanStack pipeline. The attestation wasn't lying.

This episode walks the six minutes, what SLSA was supposed to prevent versus what it actually prevents, the six-year arc from SolarWinds Orion to TanStack, and the AI cyber trilogy of the last twelve weeks that makes this the wrong moment for any of it to fail.

The attack chain minute by minute. Pwn Request via pull request target. Cache poisoning across the workflow boundary. OIDC token theft from the GitHub Actions runner. Eighty-four npm publishes in sixty seconds, every one producing valid SLSA provenance — because the publishes really came from the official TanStack workflow on the main branch on a hardened build platform. SLSA L3 verified everything it was designed to verify. It just doesn't verify that the inputs to the build script were the intended inputs.

Then the lineage. SolarWinds. Codecov. node-ipc. xz utils Jia Tan caught by a Microsoft engineer who noticed half a second of SSH latency. tj-actions. Shai-Hulud. Each moved the trust failure up the stack.

And the AI cyber trilogy. Hagendorff in Nature: LRMs jailbreaking LRMs at 97 percent ASR. UK AISI on GPT-5.5 at 71.4 percent expert cyber tasks. Google Threat Intelligence Group confirming the first criminal AI-built zero-day on the same day TanStack got hit. Frontier cyber offense doubling every 3.4 months.

Defense in six layers: SLSA provenance, Sigstore, SBOMs, OIDC trusted publishing across npm/PyPI/RubyGems/crates.io, pre-publish package analysis (the layer that actually caught TanStack), runtime detection. What an engineer can do this week: audit pull-request-target workflows, pin third-party actions to commit SHAs, namespace caches by workflow.

Plus regulation (U.S. weaker after EO 14306, EU stronger via the Cyber Resilience Act), the maintainer economics problem nobody is fixing, and five predictions.

The thesis: the frameworks help. What actually catches the next one is somebody paying attention to a five-times tarball-size anomaly.

RELATED EPISODES

The Palo Alto CVE Cluster — sister cyber episode; Klarich's 3-to-5-month window quoted on the record

The ShinyHunters SSO Breach — SaaS-side attack surface as the trust chain shifted from packages to pipelines

Claude Mythos — the model class behind Google TIG's first criminal AI-built zero-day on May 11

The AI Layoff Gap — cybersecurity tributary; security-analyst postings -25.88% as the canary cohort

CHAPTERS

00:00 Cold open — the 6-minute TanStack window

00:47 Show intro and roadmap

01:18 Callback — LiteLLM, the same pattern weeks earlier

01:58 The attack chain, minute by minute

05:56 SLSA — what it actually guarantees

08:12 The 6-year lineage — SolarWinds to TanStack

11:51 The AI cyber trilogy of the last 12 weeks

15:18 Defensive architecture, six layers

17:42 What an engineer can do this week

18:44 Regulation — U.S. weaker, EU stronger

20:25 The maintainer economics problem

22:23 Predictions and closing

SOURCES

TanStack incident postmortem (May 11, 2026)

Snyk + Socket — TanStack 42-package compromise analysis

SLSA v1.0 specification (slsa.dev)

Sigstore project documentation

Executive Orders 14028, 14144, 14306

EU Cyber Resilience Act (Regulation 2024/2847)

Sonatype — 2025 State of the Software Supply Chain

Hagendorff et al. — Nature 2026 (LRM-on-LRM jailbreak)

UK AISI — GPT-5.5 evaluation (May 7, 2026)

Google Threat Intelligence Group — first criminal AI-built 0-day (May 11, 2026)

Andres Freund — xz utils backdoor discovery (oss-security)

Tidelift — 2024 State of the Open Source Maintainer Report

Retrieval-Augmented Generation is in every production LLM application now. Most of them fail in similar, specific ways — and the fixes are mostly not about the LLM. This episode walks through the pipeline layer by layer, from chunking to embeddings to vector indexes to hybrid retrieval to reranking, with empirical numbers from two production RAG systems built for this show — including the one that caught two real factual errors in an already-published episode.

The thesis: the 80/20 of RAG quality lives in retrieval, not in the language model at the end. Anthropic's Contextual Retrieval reduced retrieval failure rate by 67 percent without touching the LLM. That's the shape of the problem.

What's actually covered. The Lewis et al. 2020 paper that named RAG, and how modern production diverges from it. Why your cosine-similarity thresholds are probably wrong (empirical distribution on text-embedding-3-small: off-topic 0.10 to 0.25, narrative match 0.50 to 0.65, sequel-grade overlap 0.65 to 0.70 — set thresholds from observed distribution, not textbook defaults). HNSW, IVF, Product Quantization — when each wins at scale, and why a billion-vector index needs 6 terabytes of RAM at full precision. Hybrid retrieval with BM25 plus dense embedding, plus reranking — Anthropic's 5.7 to 1.9 percent failure cascade as the cleanest published demonstration.

Then the production failure modes. Junk retrieval. Missing context. Hallucination on grounded generation. Stale data. Multi-document reasoning failures. Lost in the middle. And the seventh: wrong-topic evidence retrieval. The "Cheng versus Costello" pattern — the verify-claims-rag system flagged a script claim as wrong, citing evidence about a different study. The retrieval surfaced a related-but-different paper and the judge couldn't tell. Demonstrated live on the script for this episode.

RAG versus long context. Claude 4.7 at 1 million tokens. GPT-5.5 at 1 million. Gemini 2 at 2 million. The 2024 question — is RAG obsolete — has a clearer 2026 answer. No. But the line moved. RULER showed the headline 1 million-token context claims drop to roughly 60 percent effective recall on real long-document tasks even when Needle in a Haystack says 99 percent. The 2026 default architecture is compound: long context for cross-document reasoning, RAG for fresh data and citation, light fine-tuning for output format.

Plus five predictions on where the field is going through end of 2026. Companion to the show's "How LLM Inference Actually Works" — same shape, different layer.

RELATED EPISODES

How LLM Inference Actually Works — companion technical-mechanism episode (same shape, different layer)

Speculative Decoding — companion inference-optimization episode

Warm AI Sycophancy — LLM reliability lens

Cerebras IPO — the case study (this episode caught two errors there)

CHAPTERS

00:00 Cold open — two errors caught in a published episode

01:16 Today's pipeline

03:11 Chunking

05:57 Embeddings and the threshold table

08:57 Vector indexes

11:35 Hybrid retrieval and reranking

14:12 What breaks in production

17:51 Cheng-vs-Costello pattern + EP34 catches

19:26 RAG vs long context

21:17 The frontier and predictions

24:43 Closing — the trust layer

SOURCES

Lewis et al. 2020 — RAG (arxiv 2005.11401) + Karpukhin DPR + Khattab ColBERT

Malkov & Yashunin — HNSW (arxiv 1603.09320) + Cormack RRF

Anthropic — Contextual Retrieval (anthropic.com/news/contextual-retrieval)

Asai Self-RAG + Yan Corrective RAG + Edge GraphRAG

Liu Lost in the Middle + Hsieh RULER (arxiv 2404.06654)

Databricks — Long Context RAG Capabilities (Oct 2024)

Cheng 2025 sycophancy follow-up (N=1,604) + Costello 2024 DebunkBot (Science, N=2,190)

Notion — Turbopuffer migration + Klarna 2024/2025 walkback + MongoDB-Voyage AI

CVE-2026-0300. Unauthenticated remote code execution as root on Palo Alto firewalls. CVSS 9.3. Disclosed May 6, 2026. CISA added it to the Known Exploited Vulnerabilities catalog the same day and set a federal civilian patch deadline of May 9. The first patch batch ships May 13. The federal mandate predates the patch by four days.

This is a structural problem.

Binding Operational Directive 22-01 — issued November 2021 — gives federal civilian agencies two paths to KEV compliance: apply the vendor patch, or remove the product from the network. Mitigations are explicitly temporary. When CISA used the standard KEV instrument here, agencies inherited an impossible deadline. The closest historical analog is Ivanti Connect Secure in January 2024 — but there, CISA issued Emergency Directive 24-01, which explicitly accepts mitigation as compliance. Standard KEV doesn't.

And the market response is the counterintuitive twist. Palo Alto Networks stock went up after disclosure: +5.63% on May 7, +3.79% on May 8. PANW closed near 450 dollars. Three analysts raised price targets the same week. The 25 to 28 percent drop that month in 2024 was a platformization guidance cut, not the CVE that followed in April. The market has learned to price critical edge-device CVEs as routine.

This episode walks through what CVE-2026-0300 actually is (the User-ID Authentication Portal — and it is not default-on, only enabled when admins turn it on for BYOD or guest SSO; Shodan finds 67 instances exposed on port 6081 versus 225 to 263 thousand total PAN-OS deployments), what an attacker does with root on a firewall (network pivot, SSL forward-proxy key extraction, persistence surviving reset and upgrade), how Unit 42 frames attribution as CL-STA-1132 — likely state-sponsored, with EarthWorm tool reuse as inference toward Chinese-nexus actors but never confirmation — and the live policy collision: National Cyber Director Sean Cairncross and acting CISA chief Nick Andersen are debating a permanent move to three-day standard KEV deadlines at the exact moment this CVE demonstrates three-day deadlines cannot work when patches need seven or more.

Plus the pattern. Edge appliances are now the number one attack vector for state actors. Trend Micro reports the edge-device share of all exploitation incidents went from 3 percent to 22 percent in a single year — roughly an 8x increase. PAN-OS in 2024 and again in 2026. Ivanti twice in 2024 and 2025. Cisco IOS XE in 2023. Fortinet across three years. Citrix NetScaler in 2023. Same architecture. Same outcome.

When the mandate is impossible and the market doesn't care, the only thing that gets fixed is the next CVE.

RELATED EPISODES

SLSA / TanStack — sister cyber episode (trust-the-vendor failure mode)

Claude Mythos — model behind Palo Alto's 26 CVEs across 130+ products

The AI Chip War — federal cyber budget + appliance-procurement angle

CHAPTERS

00:00 Cold open — the impossible sequence

01:15 Intro

01:35 The CVE itself

07:24 What attackers do with root on a firewall

10:00 Attribution — CL-STA-1132

14:10 The mandate-then-patch gap

17:19 The market response — PANW stock went UP

19:44 The pattern — edge appliances as #1 attack vector

21:12 What defenders should do this week

22:27 Three signals to watch

23:30 Closing thesis

SOURCES

Palo Alto Security Advisory CVE-2026-0300 + Unit 42 Threat Brief CL-STA-1132

CISA KEV catalog + BOD 22-01 (Nov 2021) + FBI IC3 edge-device advisories

Reuters + SC Media + CSO Online — Cairncross/Andersen 3-day default reporting

Shadowserver + Wiz + Help Net + BleepingComputer — CVE-2026-0300 coverage

Trend Micro — 2026 edge-device exploitation share (3% → 22% YoY)

VulnCheck 2024 zero-day catalog + Five Eyes Feb 2025 advisory

Palo Alto Networks SEC filings — FY26 guidance, market share

NVIDIA bought Groq for about $20 billion on Christmas Eve 2025 — Jonathan Ross and roughly 90 percent of Groq's engineering team moved into NVIDIA in what was structured as a licensing deal. SambaNova raised a down round in February 2026 at roughly $2.2 billion, off a 2021 peak of $5.1 billion. Cerebras Systems prices its IPO this Tuesday evening at roughly a $30 billion implied valuation, trading Wednesday on Nasdaq under ticker CBRS — the largest AI-infrastructure listing since Arm in September 2023.

That makes Cerebras the last independent fast-inference pure-play in AI chips on the public markets.

The bet investors are pricing is simple. Reasoning models — OpenAI's o-series, DeepSeek R1, extended-thinking Claude — generate 10 to 100 times more tokens per query than ChatGPT-3.5 did. Memory bandwidth, not compute, becomes the binding constraint. A wafer-scale chip built around 44 GB of on-die SRAM, running Llama 70B at 2,100 tokens per second versus 30 to 100 tokens per second on H100, exploits that shift in a way no GPU cluster can mechanically match. That's why OpenAI signed a $20 billion-plus capacity agreement.

The risk is harder. Cerebras's biggest committed customer is also the customer building the chip designed to replace it. OpenAI's Titan accelerator, co-developed with Broadcom on TSMC 3nm, enters mass production in the second half of 2026 — about six months after the IPO. The 180-day insider lockup expires around November 2026, coinciding with the Titan production ramp.

This episode is the structural argument: why wafer-scale matters now, how 84 dies become one chip via custom scribe-line lithography, what the $237.8 million GAAP net income actually means (driven by a $363 million non-cash forward-contract gain — operating loss was $145.9 million), the 86 percent UAE customer concentration that migrated rather than disappeared, the circular OpenAI deal that makes the IPO possible, the Graphcore precedent ($2.8 billion peak to $500 million SoftBank sale), and three signals to watch — first-day open versus offering, Q2 earnings, and OpenAI Titan production timing.

Cerebras is being IPO'd as AI infrastructure. It may end up trading as a single-customer business. November 2026 is when we find out which.

RELATED EPISODES

The AI Chip War — anchor episode; Cerebras is the alternative-chip thesis from inside the chip war

The Real Cost of AI — the economics layer; what wafer-scale economics look like at training and inference scale

How LLM Inference Actually Works — the mechanism Cerebras hardware accelerates (memory bandwidth bottleneck)

CHAPTERS

00:00 Disclaimer — analysis, not investment advice

00:08 Cold open — last independent

01:28 Intro

01:48 Why now — reasoning models break GPU economics

04:44 The chip — wafer-scale architecture

06:56 The IPO — what the numbers say

09:20 Customer concentration — 86 percent UAE

11:12 The OpenAI deal

12:03 The circular financing structure

13:58 The existential bet — OpenAI Titan

15:25 The cautionary frame — Graphcore precedent

16:23 Three signals to watch

17:38 Closing thesis

SOURCES

Cerebras S-1 (Apr 2026, SEC EDGAR); The Information — OpenAI $20B+ MRA

Bloomberg, CNBC, Yahoo Finance — IPO mechanics; NVIDIA buys Groq ~$20B (Dec 2025)

TechCrunch — OpenAI-Cerebras tie; Graphcore $2.77B peak (Dec 2020)

Tom's Hardware — OpenAI Titan + Broadcom 10GW

Cerebras Hot Chips 2024 — WSE-3 architecture

Artificial Analysis + arXiv 2503.11698 — WSE-3 benchmarks vs H100

Reuters, SiliconANGLE — CFIUS clearance Mar 2025; Sacra — SambaNova down round

SemiAnalysis — hyperscaler captive silicon (Trainium3, TPU v7, Maia)

———

This episode is for educational and informational purposes only and does not constitute financial, investment, or trading advice, nor a recommendation to buy or sell any security. Deep Dive is not a registered investment adviser. All investing involves risk. Consult a licensed financial professional before making investment decisions.

Meta's Llama 3 training run, 405 billion parameters, used 16,384 H100 GPUs for 54 days. Over those 54 days, the cluster experienced 419 unexpected interruptions — roughly one failure every three hours. And that's the run Meta calls a success. They hit 90 percent effective training time.

This is the substrate platform engineers at AI-native companies are operating on.

This episode is what's actually different about platform engineering at companies like OpenAI and Anthropic, compared to the traditional shape — Stripe, Netflix, Block, Google. Engineering tone, not hype.

The verified primary-source view: OpenAI's two Kubernetes scaling posts at 2,500 and 7,500 nodes (5 API servers, 5 etcd, 70 GB heap per API server, 200,000 IPs in use at peak, MPI gang scheduling via the Coscheduling plugin). OpenAI's Postgres scaled for 800 million ChatGPT users on a single primary plus 50 read replicas. Anthropic's September 2025 postmortem disclosing three serving platforms (first-party, Bedrock, Vertex), three hardware backends (Trainium, NVIDIA, TPU), sticky routing, tens of chips per request.

Compute portfolios: Anthropic ~7 GW across AWS Project Rainier (~500K Trainium2), Google-Broadcom (up to 1M TPUs), Microsoft-NVIDIA ($30B / 1 GW Grace Blackwell + Vera Rubin), SpaceX Colossus 1 (220K NVIDIA / 300 MW). OpenAI Stargate at $500B / 10 GW.

The new problem classes: training cluster reliability (Meta MTTF 47.7 days at 8 GPUs → 14 minutes at 131,072 GPUs — collapses non-linearly). NCCL collectives. Gang scheduling primitives (Kueue vs Volcano). Inference at p99 (PagedAttention, RadixAttention, continuous batching). Prefill vs decode disaggregation. Heterogeneous fleets across H100, H200, B200, GB200, Trainium2, TPU v5p, Ironwood. HBM and U.S. energy as the binding constraints, not GPU FLOPS.

What stays the same: the reliability discipline. SLOs, error budgets, on-call, blameless postmortems, observability. Anthropic's September 2025 postmortem reads like a Google SRE Book chapter. What doesn't transfer: substrate-specific tooling. You can't canary a 16,000-GPU job mid-flight.

Three platforms inside one company. Training is a batch-scheduler problem. Inference is a request/response problem. Agents are a durable-workflow problem. Above all three, a chip-portability layer.

Same craft. Different physics.

RELATED EPISODES

How Netflix, Uber, and YouTube Handle Scale — sister episode, traditional platform

How LLM Inference Actually Works — the inference layer this scales

The AI Chip War — the H100/Trainium/TPU substrate platforms run on

The Real Cost of AI — economics of running this fleet at 7 gigawatts

RAG in Production — the application layer above the platform

CHAPTERS

00:00 Cold open — Llama 3.1 reliability data

00:33 Intro

00:59 The traditional platform charter

02:24 What's disclosed at OpenAI + Anthropic

04:40 Anthropic infrastructure deep dive

07:10 Team structure (OpenAI by workload, Anthropic by portability)

07:48 The new problem classes

08:20 Training cluster reliability + Meta MTTF curve

09:52 Gang scheduling — Kueue vs Volcano

10:26 Training frameworks — DeepSpeed, FSDP, Megatron

11:15 Inference at p99 — PagedAttention, RadixAttention

11:58 Prefill vs decode disaggregation

12:38 Heterogeneous fleets

13:14 Capacity planning + HBM as the binding constraint

14:28 What stays the same

15:46 Why 'more load-bearing'

16:59 Closing thesis

SOURCES

OpenAI Kubernetes posts (2018, 2021) + Postgres scaling writeup

Anthropic September 2025 postmortem

Anthropic Managed Agents + Code Execution with MCP

AWS Project Rainier, Google-Broadcom, MS-NVIDIA, SpaceX Colossus disclosures

OpenAI Stargate (January 2025)

Llama 3.1 paper + Meta cluster MTTF (arXiv 2410.21680)

DeepSeek V3 paper · vLLM PagedAttention (SOSP 2023) · SGLang RadixAttention

Latent Space — NVIDIA Dynamo team (prefill/decode disaggregation)

Google Borg paper · Netflix Tech Blog (Spinnaker, Atlas, Eureka)

Two language models running side by side are faster than one. A 60-million-parameter model drafting tokens for an 11-billion-parameter model gave Google a 2-to-3× speedup with mathematically guaranteed identical output. The smaller model is wrong about a third of the time. The bigger model only verifies in parallel. And somehow you come out ahead.

That's speculative decoding. The original paper landed the same day as ChatGPT — November 30, 2022. Today it runs inside Google Search, vLLM, TensorRT, and every major LLM serving stack on the planet.

This episode is the sequel to "How LLM Inference Actually Works." The mechanism. The four-line proof that says you cannot lose quality, ever. The Leviathan formula — three numbers (acceptance rate, draft length, cost ratio) that determine the speedup. Plug them in and you get the answer.

The architecture progression: small-LLM drafts (2022) → MEDUSA (2024, prediction heads on the target) → EAGLE (2024, predict feature vectors) → EAGLE-3 (2025, multi-layer feature fusion, 3.0-6.5×) → Lookahead Decoding (no draft model at all). Block Verification (ICLR 2025) — the original inventor still evolving the algorithm.

The honest production reality. Research papers say 5-6×. vLLM at production concurrency reports 1.2 to 2.5×. The Red Hat gpt-oss-120B benchmark hits +9.5 to 20.7 percent throughput improvement, not 3×. Acceptance rate below 0.55 turns the technique net-negative. Math at 0.518 actively hurts; code above 0.8 hits 6×+.

Two case studies: Cursor's 13× speedup from using the file you're editing as the draft (not a draft model, structural prior). Morph Fast Apply at 10,500 tokens per second on a 7B model. The whole AI-code-editor category runs on this trick.

MagicDec — counterintuitive long-context exception where speculative decoding helps MORE at larger batch.

Five testable predictions. Closing thesis: two LLMs running together are faster than one. The math is as old as ChatGPT itself. And it is the reason your AI is faster every six months.

RELATED EPISODES

How LLM Inference Actually Works — the mechanism this episode optimizes

The AI Chip War — H200/MI300X hardware substrate the benchmarks run on

The Real Cost of AI — economics of why 3× cheaper inference matters

Computer Use 45× — the cost layer above inference

RAG in Production — retrieval layer on top of the inference stack

CHAPTERS

00:00 Cold open — Two LLMs faster than one

01:10 EP2 recap — memory-bound inference

02:04 The mechanism — draft + verify

04:20 The four-line proof — why it's lossless

06:03 The Leviathan formula

07:26 Architecture progression: small-LLM → MEDUSA → EAGLE → EAGLE-3 → Lookahead

09:33 Block Verification (ICLR 2025)

10:07 Production reality — research vs serving

11:15 SpecDecode-Bench falloff + MagicDec exception

12:41 The α=0.55 floor + domain spread

13:12 Cursor 13× (file-as-draft) + Morph 10,500 tps

14:28 What spec decoding enabled (Realtime Voice, AI-code-editor)

14:59 The frontier — SSD, DFlash, speculative cascades

16:30 Five predictions

17:49 Closing thesis

SOURCES

Nov 30 2022 — Leviathan, Kalman, Matias (Google) 'Fast Inference from Transformers via Speculative Decoding'

Feb 2 2023 — Chen et al. (DeepMind) 'Accelerating LLM Decoding with Speculative Sampling'

Jan 2024 — MEDUSA paper (multiple decoding heads)

Jan 2024 — EAGLE paper (feature-level autoregression)

Mar 2025 — EAGLE-3 (NeurIPS 2025, multi-layer feature fusion)

Nov 2023 — Lookahead Decoding (LMSYS / Hao AI Lab)

ICLR 2025 — Block Verification (Leviathan co-authored)

Aug 2024 — MagicDec long-context paper

Dec 2025 — Google DFlash (block-diffusion on TPU v5p)

Apr 2026 — Red Hat gpt-oss-120B production benchmark on H200

Oct 2024 — vLLM speculative decoding blog (2.8× CNN/DailyMail at QPS=1)

May 2024 — Cursor 'Editing files at 1000 tokens/sec'

Berkeley EECS-2025-224 — Liu, 'Efficient LLM System with Speculative Decoding'

April 29, 2026: the Trump White House drafts an executive order to bring Anthropic back for federal use. May 4: the same White House is now considering a separate executive order — mandatory pre-release government vetting of frontier AI models, routed through NSA, ONCD, and DNI. Five days. The catalyst, every outlet agreed, was Mythos — the model Anthropic withheld in April.

Anthropic spent three years asking for AI regulation. They got it. From the administration that doesn't trust them.

This episode is what that paradox means.

The lobbying record is real. Anthropic publicly endorsed SB 1047. Detailed support on the company blog, August 2024. Lobbying disclosures show systematic engagement with the Biden EO, NIST AI RMF, and AISI's voluntary testing framework. The advocacy was for a specific kind of regulation — public-facing, NIST-administered, voluntary, transparent.

What's getting drafted is a different thing. NSA evaluations are classified. The Office of the National Cyber Director is not a research lab. Mandatory pre-release vetting routed through national security agencies inverts the accountability surface from public-and-adversarial to classified-and-deferential.

On the UK AI Safety Institute's capture-the-flag cyber benchmark, Mythos hit 73 percent — up from Opus 4.6 at 16 percent. RSP v3.0 dropped cyber operations from the formal Responsible Scaling framework five weeks before Mythos's preview. The withholding decision was a real product call against a real capability surface — and the catalyst for an EO Anthropic almost certainly didn't want.

Five testable predictions. Closing thesis: the regulation Anthropic asked for got built. The administration that doesn't trust Anthropic is now writing it.

RELATED EPISODES

Claude Mythos — the withheld model that triggered the EO

Mythos Bifurcation — the frontier-split context immediately preceding this EO

Mythos Banks — the Fed's earlier reckoning with the same capability surface

When AI Agents Go to Court — the regulatory-precedent parallel for AI accountability

CHAPTERS

00:00 Cold open — April 7, April 29, May 4

00:27 The Trump pre-release vetting EO

00:59 Recap for returning listeners

01:35 Anthropic asked for regulation. They got it.

02:24 Mythos capabilities — UK AISI cyber benchmark, 16% → 73%

03:29 Anthropic's lobbying history — SB 1047, AI EO, AISI, NIST

04:48 RSP v3.0 — dropping cyber ops five weeks before Mythos

07:53 Pre-release vetting — what classified evals change

09:27 National-security-flavored regulation vs civilian framework

11:37 First-access vs blocking — why the design matters

14:11 Five predictions

15:16 Closing thesis — regulation Anthropic asked for, written by people who don't trust them

SOURCES

May 4 2026 — NYT broke story (Trump considering mandatory pre-release vetting EO)

May 4-5 2026 — Axios, Bloomberg, Tom's Hardware, USNews, MSN syndication

April 29 2026 — Draft EO to bring Anthropic back for federal use

April 27 2026 — Dean Ball, WBUR On Point + Techdirt follow-up

April 8 2026 — Project Glasswing launch

April 7 2026 — Anthropic Mythos Preview announcement

February 24 2026 — Anthropic Responsible Scaling Policy v3.0

August 2024 — Anthropic public endorsement of SB 1047

Trump Day 1 — Biden AI EO rescinded

UK AI Safety Institute — capture-the-flag cyber benchmark (Mythos 73% vs Opus 4.6 16%)