SemiAnalysis Weekly

ClusterMAX 3.0 is here. Sam Harshe (@sharshe02) and Pratt Bhatt (@PrathmeshBhat19) join Jordan Nanos (@jordannanos) after months of testing 77 GPU clouds. Some moved up, some moved down.

Health checks, cache hit rates, NCCL recipes, agentic coding on clusters, a Linux kernel bug, neocloud security, SLAs and financing, NVIDIA's backstops, five-year-old H100s, and hosted RL training.

Read the full ClusterMAX 3.0 report: https://newsletter.semianalysis.com/p/clustermax-30-the-industry-standard
ClusterMAX inquiries: [email protected]

0:00 Cold Open

1:32 The Rankings

6:58 Health Checks

13:26 Endpoint Cache Misses

16:40 Networking and NCCL

22:52 Vibe-Coded Infra

31:27 Bug Hunting

38:36 Security

45:43 Financing Neoclouds

56:33 Hosted Training and RL

More than 300 towns, cities, and counties have voted to pause data centers in the past 18 months. Maya Barkin (@mayabarkin) and Reyk Knühtsen (@robotknower) from the SemiAnalysis Data Center, Energy, and Industrials team join Jordan Nanos (@jordannanos) to explain what a moratorium actually is, how they mapped every one of them against the US project pipeline, and why only three projects are being delayed.

Plus: the SpaceX de-annexation in Brownsville, New York vs Texas, why politicians love a moratorium, and the surprising amount of alpha in anti-data center Facebook groups. Jeremie Eliahou Ontiveros (@JeremieEO) drops in from Bangkok.

Read More: https://newsletter.semianalysis.com/p/everyone-says-datacenter-moratoriums

0:00 Intro

0:36 What Is a Moratorium

4:34 300 Moratoriums, 3 Projects

10:18 Behind the Meter Wins

14:04 Eight Conditions to Delay

17:06 The SpaceX De-Annexation

20:03 Cheap Political Signaling

32:03 New York vs Texas

41:14 City Council Alpha

46:32 Advice for Developers

Emergency episode. Doug O'Laughlin (@Fabknowledge) , Joey Brookhart (@SaasquatchC), and Max Kan (@maxkan) join Jordan (@JordanNanos) to talk through Dario's "We Must Pace the Frontier" blog and the responses from Sam Altman and David Sacks. Does pacing actually mean less compute, or a lot more? Plus the Hugging Face incident, Moonshot serving Claude, and the security behaviour that keeps them up at night.

0:00 Cold Open

2:50 Glorified Neoclouds

8:21 Pacing the Frontier

12:53 Safety Eats Compute

20:38 Hugging Face Lessons

30:52 Moonshot Serving Claude

36:37 The Coxson Resignation

38:27 Two-Year Predictions

49:19 Regulation Under Trump

57:21 Hotter or Cooler

NVIDIA previewed Rubin Ultra at 1TB of HBM per package. The part that ships will carry 192GB.

Myron Xie and Jordan Nanos get into how the design fell that far, why the reason is supply and not performance, and why shipping less memory per chip might be the right call anyway.

For the first time in recent memory, NVIDIA's next flagship will have less capacity than the one it replaces.

Read More: https://newsletter.semianalysis.com/p/long-live-the-short-king-why-4-hi

Traditional data center builds ran 18 to 24 months, sometimes past three years. Modular construction cuts that to 12. Jordan Nanos (@JordanNanos) sits down with Eric Wen, Nigel Chiang, and Nico Bontigui to map what modular actually means, from MEP heavy skids to full containerized builds.

They break down why the bottleneck moved from construction time to labor, why welders and electricians are the real constraint in Abilene, and how time to power translates directly into tokens at 40 to 100 million dollars per megawatt.

Subscribe for weekly SemiAnalysis coverage on data center construction, prefabrication, and AI infrastructure.

0:00 Cold Open

1:09 What Is Modular

4:03 Why Go Modular

5:55 Time to Power Math

9:04 Labor Wage Data

13:51 Two Kinds of Bottlenecks

18:24 Site, Shell, System

22:31 Who Owns the Risk

26:59 Trucks and Insurance

30:53 Who Supplies the Modules

35:50 Commissioning Reality

41:39 Vendor Map and Wrap

Read More: The Wild Wild West Of LEGO Datacenters: https://newsletter.semianalysis.com/p/the-wild-wild-west-of-lego-datacenters

This week Doug, Sam and Jordan discuss the OpenAI vs HuggingFace security incident and our recent article on Neocloud security ahead of ClusterMAX 3.0

Full article: https://newsletter.semianalysis.com/p/most-neoclouds-suck-at-security

0:00 Cold Open

0:57 Neocloud Security

4:38 The Hugging Face Hack

11:47 Agent Swarm Behavior

14:47 Obliterated Models

20:37 Security as a Service

24:30 What the Data Shows

33:08 Attacker Asymmetry

40:55 Nothing Ever Happens

45:22 CMAX Audit

This week Bryan, Myron and Jordan discuss our recent article on OpenAI Jalapeño. They cover the performance, architecture, programming model, implications for NVIDIA and more.1:05 Jalapeno Overview4:22 Tokens Per Megawatt9:15 Benchmark Caveats13:48 The CUDA Moat21:12 How OpenAI Did It32:13 Samsung HBM442:10 AI-Designed Silicon49:24 Architecture Deep Dive56:58 Doom and Wrap

Full Article: ⁠https://newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia

PJM overpaid $12 billion across two capacity auctions, and the same modeling error is set to repeat in an upcoming emergency auction. Robert Boswall (@RobertBoswall) and Jordan Nanos (@JordanNanos) break down how the largest grid in America, 13 states and 66 million people, inflates demand while constraining supply.

The result is scarcity pricing that ratepayers absorb, not the data centers driving the narrative. Of the $63 billion spent across four auctions, Boswell estimates $12 billion was avoidable, split $7 billion and $5 billion across the 2025-26 and 2026-27 auctions. This is the mechanics behind every headline blaming AI for rising power bills.

Subscribe for weekly analysis on grid design, capacity markets, and AI energy demand!

0:00 Cold Open1:33 What Is PJM3:13 Capacity Auctions7:39 The $12 Billion12:33 How Plants Get Paid15:15 The Winterization Gap21:17 Data Center Demand26:41 The Emergency Auction30:00 Board Overrules Members40:32 Turning It Around

Article: https://open.substack.com/pub/semianalysis/p/12b-of-us-ratepayers-money-wasted

Dylan joins the pod today, recorded live from our SF office. Jordan and Dylan discuss SemiAnalysis's AI spend, the model that escaped during training, AI rollups eating private equity, new accelerators vs NVIDIA, and whether the pod gives away too much information for free.0:00 Cold Open5:25 SA AI Spend9:29 AI Performance Reviews10:40 AI Rollups13:47 The Agent Moment15:28 The Escaped Model18:11 Model Exponential22:52 Compute and Chips30:18 Open Models32:00 ADHD and AI

OpenAI and Anthropic are adding close to $30 billion of ARR per month, and the driver is gross margin expansion, not new compute. Jeremie Eliahou Ontiveros (@JeremieEO) and Reyk Knuhtsen (@robotknower) join Jordan Nanos (@JordanNanos) to trace the $100 million per megawatt per year figure from real inference workloads on GB200 and GB300 up through SemiAnalysis simulation. The Google deal prices GB300 capacity near $14 an hour against a $3 average, a premium justified by a 90 day cancellation clause and the ability to turn on megawatts immediately.

From there the conversation moves to SpaceX's 10 gigawatt ambition, the sites and supply chain needed to hit it, the permitting playbook, and why Microsoft becomes the largest offtaker. The bull and bear cases both get airtime, followed by a Hugging Face security scare to close. Subscribe for weekly coverage of tokenomics, datacenter energy, and AI compute economics.

CHAPTERS:

0:00 – Intro
0:47 – The $100M Thesis
3:59 – Pricing & Google's Deal
10:38 – Training vs Inference
13:21 – Sites & Supply Chain
20:55 – Permitting Playbook
25:40 – Microsoft's Role
33:31 – Paying For It
41:01 – Bull vs Bear
43:29 – Hugging Face Security Scare & Wrap-Up

Referenced:
SpaceX 10GW in 2027 – Why It's Real, Will Drive $300B ARR for SpaceX, and Why Microsoft Will Be the Largest Offtaker: https://newsletter.semianalysis.com/p/spacex-10gw-in-2027-why-its-real