Last week we argued the $25 token was dying. This week: that's exactly when the hardware boom starts.
Frank takes the hot seat with a paradox. If intelligence keeps getting cheaper, we don't use less of it — we invent more things to do with it. More agents, more inference, more workloads. Which leaves every enterprise with one question they haven't had to ask before: which intelligence should we rent, and which should we own?
Citi and Vercel are reportedly running open-weight models more than half the time. In June it was 29%. By late August, 53%. A new family of models out of the UAE ships six versions built for six classes of hardware — phone to server — with no quantization involved. NVIDIA's Hugging Face acquisition is official. And OpenAI shipped Astra.
We also had four people instead of five, a guest from London, and for the first time in three episodes, a completely unanimous vote.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CHAPTERS
00:00 If intelligence gets cheap, what do you buy?
00:47 Welcome back — who's here, who's at Disneyland
01:28 Laura Osborn joins from London
02:14 Neil, and the Bears/Seahawks problem
03:07 Chauncey, and the disclosure
03:43 The Rundown: NVIDIA and Hugging Face is official
04:25 Neil: what GitHub can tell us about this
05:48 Citi and Vercel pass 50% open weights
07:50 The three-tier model: endpoint, edge, cloud
08:56 Six models for six classes of hardware
10:22 Purpose-built beats shrinking things down?
11:21 What UK and European customers actually ask for
14:41 From cloud architect to local AI
15:18 OpenAI ships Astra
16:26 Chauncey: be careful what you point it at
17:51 Neil: code generation at the edge
19:33 Baseline: what should people actually care about?
20:20 Open weights are no longer philosophical
21:27 What's coming out of IFA
22:52 The medical school of the future
25:00 Why doesn't anyone know this is possible yet?
26:53 "We have more agents than human employees"
28:23 Tension of the Week: the hardware boom begins
30:27 Chauncey: fixed cost beats a variable one
32:39 Laura: I spent five years telling everyone to go cloud
34:18 Neil: when pennies turn into a million dollars
35:00 The brewery test
36:06 The data center next door
36:34 The vote
38:42 On My Device: the 36B gets the job
40:07 Chauncey: Copilot CLI, running locally
41:44 The Dream Team and the one-person virtual business
41:56 Laura: an on-device speaker coach
42:58 Neil: Mac vs the Beast, round two
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
WHAT CAME UP
• NVIDIA acquiring Hugging Face is confirmed. NVIDIA says it stays open and that NVIDIA hardware won't be required. Neil's read: this looks like Microsoft buying GitHub — the same fears, and possibly the same outcome. His angle is that Hugging Face has been effectively off-limits to a lot of enterprises, and being inside a vendor with a security and compliance apparatus may be what finally gets those models sanctioned.
• Citi and Vercel are reported to be running open-weight models for more than half their AI workloads — 29% in June, 53% by late August. Ten weeks.
• The MBZUAI Institute of Foundational Models released six models spanning roughly 1B to 375B parameters, each trained for a specific class of hardware rather than quantized down from something larger. Frank ran the 36B on his dual-3090 workstation against Qwen3.8-27B: about 68 tokens/sec against 40, on a comparable qualifying profile.
• Neil's three tiers: the endpoint, the cloud, and an emerging middle — departmental edge servers. Local doesn't grant you HIPAA or FERPA compliance by itself, but no round trip to the cloud is a smaller attack surface.
• The economics nobody models: an insurance agent filing 50 claims a day, times 10,000 agents. Even at pennies per call, that's over a million dollars a year in workloads that could run at the edge.
• Laura, five years a cloud architect, on doing a full 180: the constraint in Europe isn't just sovereignty rules, it's that cloud capacity is at its limits.
• And Neil's field test of public sentiment, conducted on a bartender who asked him point blank whether he was anti-AI.
The vote was 4–0. First unanimous verdict of the series.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
THIS WEEK'S CREW
Frank Buchholz — host, independent
Laura Osborn — guest, joining from London
Neil Misak
Chauncey Larsen
Jacob Rhoades and Robert Henry are off this week. Jacob is celebrating his first anniversary. Robert was on a plane, sending Teams messages anyway.
DISCLOSURE: Laura, Neil and Chauncey are employed by Microsoft. Frank is independent. All views expressed are their own, nothing discussed is unannounced or non-public, and none of this is financial advice.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
localhost is a weekly show about running AI on hardware you own. Independent and unsponsored.
New episodes weekly: https://thelocalhost.show
There's no place like 127.0.0.1.
#LocalAI #OpenWeights #AI #EdgeAI #AIHardware #Sovereignty #OnDeviceAI #AIPodcast
