LessWrong posts by zvi

The big release this week was OpenAI giving us a new browser, called Atlas.

The idea of Atlas is that it is Chrome, except with ChatGPT integrated throughout to let you enter agent mode and chat with web pages and edit or autocomplete text, and that will watch everything you do and take notes to be more useful to you later.

From the consumer standpoint, does the above sound like a good trade to you? A safe place to put your trust? How about if it also involves (at least for now) giving up many existing Chrome features?

From OpenAI's perspective, a lot of that could have been done via a Chrome extension, but by making a browser some things get easier, and more importantly OpenAI gets to go after browser market share and avoid dependence on Google.

I’m going to stick with using Claude [...]

---

Outline:

(02:01) Language Models Offer Mundane Utility

(03:07) Language Models Don't Offer Mundane Utility

(04:52) Huh, Upgrades

(05:24) On Your Marks

(10:15) Language Barrier

(12:50) From ChatGPT, a Chinese answer to the question about which qualities children should have:

(13:30) ChatGPT in English on the same question:

(15:17) Choose Your Fighter

(17:19) Get My Agent On The Line

(18:54) Fun With Media Generation

(23:09) Copyright Confrontation

(25:19) You Drive Me Crazy

(35:31) They Took Our Jobs

(44:06) A Young Lady's Illustrated Primer

(44:42) Get Involved

(45:33) Introducing

(47:15) In Other AI News

(48:30) Show Me the Money

(51:03) So You've Decided To Become Evil

(53:18) Quiet Speculations

(56:11) People Really Do Not Like AI

(57:18) The Quest for Sane Regulations

(01:00:55) Alex Bores Launches Campaign For Congress

(01:03:33) Chip City

(01:10:17) The Week in Audio

(01:13:00) Rhetorical Innovation

(01:16:17) Don't Take The Bait

(01:27:29) Do You Feel In Charge?

(01:29:30) Tis The Season Of Evil

(01:34:45) People Are Worried About AI Killing Everyone

(01:36:00) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was very clearly one of those. So here we go.

As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary.

If I am quoting directly I use quote marks, otherwise assume paraphrases.

Rather than worry about timestamps, I’ll use YouTube's section titles, as it's not that hard to find things via the transcript as needed.

This was a fun one in many places, interesting throughout, frustrating in similar places to where other recent Dwarkesh interviews have been frustrating. It gave me a lot of ideas, some of which might even be good.

AGI Is Still a Decade Away

---

Outline:

(00:58) AGI Is Still a Decade Away

(12:13) LLM Cognitive Deficits

(15:31) RL Is Terrible

(17:24) How Do Humans Learn?

(24:17) AGI Will Blend Into 2% GDP Growth

(29:28) ASI

(37:38) Evolution of Intelligence and Culture

(38:57) Why Self Driving Took So Long

(42:06) Future Of Education

(48:27) Reactions

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We have the classic phenomenon where suddenly everyone decided it is good for your social status to say we are in an ‘AI bubble.’

Are these people short the market? Do not be silly. The conventional wisdom response to that question these days is that, as was said in 2007, ‘if the music is playing you have to keep dancing.’

So even with lots of people newly thinking there is a bubble the market has not moved down, other than (modestly) on actual news items, usually related to another potential round of tariffs, or that one time we had a false alarm during the DeepSeek Moment.

So, what's the case we’re in a bubble? What's the case we’re not?

My Answer In Brief

People get confused about bubbles, often applying that label any time prices fall. So you have to be clear on what [...]

---

Outline:

(01:17) My Answer In Brief

(02:18) Time Sensitive Point of Order: Alex Bores Launches Campaign For Congress, If You Care About AI Existential Risk Consider Donating

(04:09) So They're Saying There's a Bubble

(05:04) AI Is Atlas And People Worry It Might Shrug

(05:35) Can A Bubble Be Common Knowledge?

(08:33) Steamrollers, Picks and Shovels

(09:27) What Can Go Up Must Sometimes Go Down

(11:36) What Can Go Up Quite A Lot Can Go Even More Down

(13:17) Step Two Remains Important

(15:00) Oops We Might Do It Again

(15:49) Derek Thompson Breaks Down The Arguments

(17:47) AI Revenues Are Probably Going To Go Up A Lot

(20:19) True Costs That Matter Are Absolute Not Relative

(21:05) We Are Spending a Lot But Also Not a Lot

(22:46) Valuations Are High But Not Super High

(23:59) Official GPU Depreciation Schedules Seem Pretty Reasonable To Me

(29:14) The Bubble Case Seems Weak

(30:53) What It Would Mean If Prices Did Go Down

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As usual when things split, Part 1 is mostly about capabilities, and Part 2 is mostly about a mix of policy and alignment.

Table of Contents

---

Outline:

(00:20) The Quest for Sane Regulations

(05:56) People Really Dislike AI

(12:22) Chip City

(13:12) The Week in Audio

(13:24) Rhetorical Innovation

(20:53) Water Water Everywhere

(23:57) Read Jack Clark's Speech From The Curve

(28:26) How One Other Person Responded To This Thoughtful Essay

(38:43) A Better Way To Disagree

(59:39) Voice Versus Exit

(01:03:51) The Dose Makes The Poison

(01:06:44) Aligning a Smarter Than Human Intelligence is Difficult

(01:10:08) You Get What You Actually Trained For

(01:15:54) Messages From Janusworld

(01:18:37) People Are Worried About AI Killing Everyone

(01:22:40) The Lighter Side

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Well, one person says ‘demand,’ another says ‘give the thumbs up to’ or ‘welcome our new overlords.’ Why quibble? Surely we’re all making way too big a deal out of this idea of OpenAI ‘treating adults like adults.’ Everything will be fine. Right?

Why not focus on all the other cool stuff happening? Claude Haiku 4.5 and Veo 3.1? Walmart joining ChatGPT instant checkout? Hey, come back.

Alas, the mass of things once again got out of hand this week, so we’re splitting the update into two parts.

Table of Contents

---

Outline:

(00:51) Earlier This Week

(02:25) Language Models Offer Mundane Utility

(05:44) Language Models Don't Offer Mundane Utility

(11:47) Huh, Upgrades

(14:55) We Patched The Torment Nexus, Turn It Back On

(27:49) On Your Marks

(34:33) Choose Your Fighter

(35:50) Deepfaketown and Botpocalypse Soon

(36:15) Fun With Media Generation

(40:32) Copyright Confrontation

(41:04) AIs Are Often Absurd Sycophants

(44:32) They Took Our Jobs

(54:04) Find Out If You Are Worried About AI Killing Everyone

(55:42) A Young Lady's Illustrated Primer

(01:02:59) AI Diffusion Prospects

(01:11:13) The Art of the Jailbreak

(01:14:38) Get Involved

(01:14:57) Introducing

(01:16:48) In Other AI News

(01:18:22) Show Me the Money

(01:20:45) Quiet Speculations

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It is increasingly often strange compiling the monthly roundup, because life comes at us fast. I look at various things I’ve written, and it feels like they are from a different time. Remember that whole debate over free speech? Yeah, that was a few weeks ago. Many such cases. Gives one a chance to reflect.

In any case, here we go.

Table of Contents

---

Outline:

(00:37) Don't Provide Bad Training Data

(02:25) Maybe Don't Say Maybe

(03:00) Throwing Good Parties Means Throwing Parties

(05:06) Air Travel Gets Worse

(06:16) Bad News

(09:03) You Do Not Need To Constantly Acknowledge That There Is Bad News

(12:52) Prediction Market Madness

(16:00) No Reply Necessary

(19:49) While I Cannot Condone This

(23:42) Antisocial Media

(29:22) Government Working

(47:12) Tylenol Does Not Cause Autism

(51:12) Jones Act Watch

(52:13) For Science!

(54:29) Work Smart And Hard

(55:34) So Emotional

(59:17) Where Credit Is Due

(01:01:30) Good News, Everyone

(01:03:37) I Love New York

(01:08:14) For Your Entertainment

(01:17:14) Gamers Gonna Game Game Game Game Game

(01:19:53) I Was Promised Flying Self-Driving Cars

(01:24:33) Sports Go Sports

(01:29:06) Opportunity Knocks

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What is going on with, and what should we do about, the Chinese declaring extraterritorial exports controls on rare earth metals, which threaten to go way beyond semiconductors and also beyond rare earths into things like lithium and also antitrust investigations?

China also took other actions well beyond only rare Earths, including going after Qualcomm, lithium and everything else that seemed like it might hurt, as if they are confident that a cornered Trump will fold and they believe they have escalation dominance and are willing to use it.

China now has issued reassurances that it will allow all civilian uses of rare earths and not to worry, but it seems obvious that America cannot accept a Chinese declaration of extraterritorial control over entire world supply chains, even if China swears it will only narrowly use that power. In response, Trump has threatened massive tariffs and cancelled [...]

---

Outline:

(01:20) Was This Provoked?

(02:43) What Is China Doing?

(03:37) How Is America Responding?

(05:58) How Is China Responding To America's Response?

(07:14) What To Make Of China's Attempted Reassurances?

(08:15) How Should We Respond From Here?

(09:37) It Looks Like China Overplayed Its Hand

(10:36) We Need To Mitigate China's Leverage Across The Board

(13:11) What About The Chip Export Controls?

(14:01) This May Be A Sign Of Weakness

(15:06) What Next?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

A little over a month ago, I documented how OpenAI had descended into paranoia and bad faith lobbying surrounding California's SB 53.

This included sending a deeply bad faith letter to Governor Newsom, which sadly is par for the course at this point.

It also included lawfare attacks against bill advocates, including Nathan Calvin and others, using Elon Musk's unrelated lawsuits and vendetta against OpenAI as a pretext, accusing them of being in cahoots with Elon Musk.

Previous reporting of this did not reflect well on OpenAI, but it sounded like the demand was limited in scope to a supposed link with Elon Musk or Meta CEO Mark Zuckerberg, links which very clearly never existed.

Accusing essentially everyone who has ever done anything OpenAI dislikes of having united in a hallucinated ‘vast conspiracy’ is all classic behavior for OpenAI's Chief Global Affairs Officer Chris Lehane [...]

---

Outline:

(02:35) What OpenAI Tried To Do To Nathan Calvin

(07:22) It Doesn't Look Good

(10:17) OpenAI's Jason Kwon Responds

(19:14) A Brief Amateur Legal Analysis Of The Request

(21:33) What OpenAI Tried To Do To Tyler Johnston

(25:50) Nathan Compiles Responses to Kwon

(29:52) The First Thing We Do

(36:12) OpenAI Head of Mission Alignment Joshua Achiam Speaks Out

(40:16) It Could Be Worse

(41:31) Chris Lehane Is Who We Thought He Was

(42:50) A Matter of Distrust

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The 2025 State of AI Report is out, with lots of fun slides and a full video presentation. They’ve been consistently solid, providing a kind of outside general view.

I’m skipping over stuff my regular readers already know that doesn’t bear repeating.

Qwen The Fine Tune King For Now

Nathan Benaich: Once a “Llama rip-off,” @Alibaba_Qwen now powers 40% of all new fine-tunes on @huggingface. China's open-weights ecosystem has overtaken Meta's, with Llama riding off into the sunset…for now.

I highlight this because the ‘for now’ is important to understand, and to note that it's Qwen not DeepSeek. As in, models come and models go, and especially in the open model world people will switch on you on a dime. Stop worrying about lock-ins and mystical ‘tech stacks.’

Rise Of The Machines

Robots now reason too. “Chain-of-Action” planning brings structured thought to the [...]

---

Outline:

(00:39) Qwen The Fine Tune King For Now

(01:19) Rise Of The Machines

(01:40) Model Context Protocol Wins Out

(02:06) Benchmarks Are Increasingly Not So Useful

(02:56) The DeepSeek Moment Was An Overreaction

(03:40) Capitalism Is Hard To Pin Down

(04:30) The Odds Are Against Us And The Situation Is Grim

(06:35) They Grade Last Year's Predictions

(09:35) Their Predictions for 2026

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI is making deals and shipping products. They locked in their $500 billion valuation and then got 10% of AMD in exchange for buying a ton of chips. They gave us the ability to ‘chat with apps’ inside of ChatGPT. They walked back their insane Sora copyright and account deletion policies and are buying $50 million in marketing. They’ve really got a lot going on right now.

Of course, everyone else also has a lot going on right now. It's AI. I spent the last weekend at a great AI conference at Lighthaven called The Curve.

The other big news that came out this morning is that China is asserting sweeping extraterritorial control over rare earth metals. This is likely China's biggest card short of full trade war or worse, and it is being played in a hugely escalatory way that America obviously can’t accept. Presumably this [...]

---

Outline:

(01:33) Language Models Offer Mundane Utility

(03:39) Language Models Don't Offer Mundane Utility

(05:13) Huh, Upgrades

(08:21) Chat With Apps

(11:32) On Your Marks

(12:51) Choose Your Fighter

(13:47) Fun With Media Generation

(24:03) Deepfaketown and Botpocalypse Soon

(27:46) You Drive Me Crazy

(31:33) They Took Our Jobs

(34:35) The Art of the Jailbreak

(34:55) Get Involved

(38:31) Introducing

(42:18) In Other AI News

(43:38) Get To Work

(45:45) Show Me the Money

(50:24) Quiet Speculations

(52:15) The Quest for Sane Regulations

(01:02:31) Chip City

(01:06:53) The Race to Maximize Rope Market Share

(01:08:49) The Week in Audio

(01:09:28) Rhetorical Innovation

(01:10:13) Paranoia Paranoia Everybody's Coming To Test Me

(01:13:10) Aligning a Smarter Than Human Intelligence is Difficult

(01:14:12) Free Petri Dish

(01:15:45) Unhobbling The Unhobbling Department

(01:17:46) Serious People Are Worried About Synthetic Bio Risks

(01:19:09) Messages From Janusworld

(01:28:54) People Are Worried About AI Killing Everyone

(01:32:14) Other People Are Excited About AI Killing Everyone

(01:40:48) So You've Decided To Become Evil

(01:45:25) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.