LessWrong posts by zvi

Last week had the release of GPT-5.1, which I covered on Tuesday.

This week included Gemini 3, Nana Banana Pro, Grok 4.1, GPT 5.1 Pro, GPT 5.1-Codex-Max, Anthropic making a deal with Microsoft and Nvidia, Anthropic disrupting a sophisticated cyberattack operation and what looks like an all-out attack by the White House to force through a full moratorium on and preemption of any state AI laws without any substantive Federal framework proposal.

Among other things, such as a very strong general analysis of the relative position of Chinese open models. And this is the week I chose to travel to Inkhaven. Whoops. Truly I am now the Matt Levine of AI, my vacations force model releases.

Larry Summers resigned from the OpenAI board over Epstein, sure, why not.

So here's how I’m planning to handle this, unless something huge happens.

---

Outline:

(01:50) Language Models Offer Mundane Utility

(02:43) Tool, Mind and Weapon

(06:55) Choose Your Fighter

(07:18) Language Models Don't Offer Mundane Utility

(11:31) First Things First

(12:12) Grok 4.1

(15:03) Misaligned?

(18:37) Codex Of Ultimate Coding

(20:21) Huh, Upgrades

(20:49) On Your Marks

(22:11) Paper Tigers

(26:26) Overcoming Bias

(31:11) Deepfaketown and Botpocalypse Soon

(31:45) Fun With Media Generation

(33:41) A Young Lady's Illustrated Primer

(38:31) They Took Our Jobs

(44:25) On Not Writing

(44:51) Get Involved

(45:46) Introducing

(48:29) In Other AI News

(52:30) Anthropic Completes The Trifecta

(54:08) We Must Protect This House

(59:12) AI Spy Versus AI Spy

(01:05:10) Show Me the Money

(01:08:01) Bubble, Bubble, Toil and Trouble

(01:11:14) Quiet Speculations

(01:12:17) The Amazing Race

(01:17:37) Of Course You Realize This Means War (1)

(01:21:30) The Quest for Sane Regulations

(01:23:42) Chip City

(01:24:31) Of Course You Realize This Means War (2)

(01:30:02) Samuel Hammond on Preemption

(01:36:17) Of Course You Realize This Means War (3)

(01:44:43) The Week in Audio

(01:45:47) It Takes A Village

(01:46:26) Rhetorical Innovation

(01:49:53) Varieties of Doom

(01:50:59) The Pope Offers Wisdom

(01:52:46) Aligning a Smarter Than Human Intelligence is Difficult

(01:54:51) Messages From Janusworld

(02:02:05) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Happy Gemini Week to those who celebrate. Coverage of the new release will begin on Friday. Meanwhile, here's this month's things that don’t go anywhere else.

Good News, Everyone

Google has partnered with Polymarket to include Polymarket odds into Google Search and Google Finance. This is fantastic and suggests we should expand the number of related markets on Polymarket.

In many ways Polymarket prediction markets are remarkably accurate, but here what we have is a Brier Score without a baseline of what we should expect as a baseline. You need to compare your Brier Score to scores on exactly the same events, or it doesn’t mean much. There's a lot to be made on Polymarket if you pay attention.

A proposed ‘21st Century Civilization Curriculum’ for discussion groups. There's an interestingly high number of book reviews involved as opposed to the actual books. I get one post in at the end, which turns out to be Quotes From Moral Mazes, so I’m not sure it counts but the curation is hopefully doing important work there.

Wylfa in North Wales will host the UK's first small modular nuclear reactors, government to invest 2.5 billion.

[...]

---

Outline:

(00:23) Good News, Everyone

(01:40) Good Advice

(02:45) Where's The Party?

(03:04) Antisocial Media

(06:28) Government Working

(17:14) Jones Act Watch

(17:54) Variously Effective Altruism

(20:56) Great Taste, Less Filling

(24:14) How You Do Anything

(25:13) Bad News

(28:34) The Rage of the Plastic Straw Ban

(30:55) Affordability Politics

(31:30) Monks In The Casino

(32:05) Procrastination Is Bad Actually

(32:55) Many Successful People Adjust Behaviors More

(34:43) Take The Money And Run

(36:50) Work Harder

(37:26) Work Smarter

(38:22) Anti-Suicide Chairs

(38:51) The Great American Songbook

(42:04) For Your Entertainment

(44:14) The Subscription Package Dance

(45:51) Was Television Better Before?

(47:52) The Joys Of Partial Task Automation

(48:36) Gamers Gonna Game Game Game Game Game

(51:35) Sports Go Sports

(01:01:11) Sometimes People Cheat At Poker

(01:02:37) Opportunity Knocks

(01:03:28) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In honor of my dropping by Inkhaven at Lighthaven in Berkeley this week, I figured it was time for another writing roundup. You can find #1 here, from March 2025.

I’ll be there from the 17th (the day I am publishing this) until the morning of Saturday the 22nd. I am happy to meet people, including for things not directly about writing.

Table of Contents

How I Use AI For Writing These Days

How have I been using AI in my writing?

Directly? With the writing itself? Remarkably little. Almost none.

I am aware that this is not optimal. But at current capability levels, with the prompts and tools I know [...]

---

Outline:

(00:33) How I Use AI For Writing These Days

(03:44) Influencing Influence

(05:34) Size Matters

(06:23) Time To Write A Shorter One

(07:51) A Useful Tool

(08:08) A Maligned Tool

(10:04) Neglected Topics

(12:31) The Humanities Don't Seem Relevant To Writing About Future Humanity?

(13:50) Writing Every Day

(14:20) Writing As Deep Work

(15:25) Most Of Your Audience Is Secondhand

(17:57) That's Funny

(18:38) Fiction Writing Advice

(19:13) Just Say The Thing

(23:13) Cracking the Paywall

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

There are other model releases to get to, but while we gather data on those, first things first. OpenAI has given us GPT-5.1: Same price including in the API, Same intelligence, better mundane utility?

Their Announcement

Sam Altman (CEO OpenAI): GPT-5.1 is out! It's a nice upgrade.

I particularly like the improvements in instruction following, and the adaptive thinking.

The intelligence and style improvements are good too.

Also, we’ve made it easier to customize ChatGPT. You can pick from presets (Default, Friendly, Efficient, Professional, Candid, or Quirky) or tune it yourself.

OpenAI: GPT-5.1 in ChatGPT is rolling out to all users this week.

It's smarter, more reliable, and a lot more conversational.

GPT-5.1 is now better at:

– Following custom instructions

– Using reasoning for more accurate responses

– And just better at chatting overall

GPT-5.1 Instant is now warmer and more conversational.

The model can use adaptive reasoning to decide to think a bit longer before responding to tougher questions.

It also has improved instruction following, so the model more reliably answers the question you actually asked.

GPT-5.1 Thinking now more effectively adjusts [...]

---

Outline:

(00:26) Their Announcement

(04:36) Their Pitch on GPT-5.1 Instant

(06:39) Their Pitch on GPT-5.1 Thinking

(09:15) Now With Extra Glaze

(14:16) Genuine People Personalities

(15:24) The End Of The Em-Dash?

(17:15) Turning A Dial And Looking Back At The Audience

(18:00) System Card

(19:12) On Your Marks

(19:58) Ask Them Anything

(21:55) Reactions Introduction

(22:24) Officially Pitched Developer Reactions

(24:47) Positive Reactions

(30:57) Personality Reactions

(33:46) Verbosity Reactions

(35:05) Negative Reactions

(37:22) Initial Pliny Report

(39:03) The #Keep4o Crowd Is Not Happy, Defends 5.0

(42:39) Overall Take

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

GPT-4o has been a unique problem for a while, and has been at the center of the bulk of mental health incidents involving LLMs that didn’t involve character chatbots. I’ve previously covered related issues in AI Craziness Mitigation Efforts, AI Craziness Notes, GPT-4o Responds to Negative Feedback, GPT-4o Sycophancy Post Mortem and GPT-4o Is An Absurd Sycophant. Discussions of suicides linked to AI previously appeared in AI #87, AI #134, AI #131 Part 1 and AI #122.

The Latest Cases Look Quite Bad For OpenAI

I’ve consistently said that I don’t think it's necessary or even clearly good for LLMs to always adhere to standard ‘best practices’ defensive behaviors, especially reporting on the user, when dealing with depression, self-harm and suicidality. Nor do I think we should hold them to the standard of ‘do all of the maximally useful things.’

Near: while the llm response is indeed really bad/reckless its worth keeping in mind that baseline suicide rate just in the US is ~50,000 people a year; if anything i am surprised there aren’t many more cases of this publicly by now

I do think it's fair to insist they never actively encourage suicidal behaviors.

---

Outline:

(00:47) The Latest Cases Look Quite Bad For OpenAI

(02:34) Routing Sensitive Messages Is A Dominated Defensive Strategy

(03:28) Some 4o Users Get Rather Attached To The Model

(05:04) A Theory Of How All This Works

(07:38) Maybe This Is Net Good In Spite Of Everything?

(11:10) Could One Make A 'Good 4o'?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Pope offered us wisdom, calling upon us to exercise moral discernment when building AI systems. Some rejected his teachings. We mark this for future reference.

The long anticipated Kimi K2 Thinking was finally released. It looks pretty good, but it's too soon to know, and a lot of the usual suspects are strangely quiet.

GPT-5.1 was released yesterday. I won’t cover that today beyond noting it exists, so that I can take the time to properly assess what we’re looking at here. My anticipation is this will be my post on Monday.

I’m also going to cover the latest AI craziness news, including the new lawsuits, in its own post at some point soon.

In this post, among other things: Areas of agreement on AI, Meta serves up scam ads knowing they’re probably scam ads, Anthropic invests $50 billion, more attempts to assure you that your life won’t change despite it being obvious this isn’t true, and warnings about the temptation to seek out galaxy brain arguments.

A correction: I previously believed that the $500 billion OpenAI valuation did not include the nonprofit's remaining equity share. I have been informed this is incorrect [...]

---

Outline:

(01:31) Language Models Offer Mundane Utility

(04:58) Language Models Don't Offer Mundane Utility

(06:31) Huh, Upgrades

(07:06) On Your Marks

(07:37) Copyright Confrontation

(08:59) Deepfaketown and Botpocalypse Soon

(11:07) Fun With Media Generation

(13:48) So You've Decided To Become Evil

(18:45) They Took Our Jobs

(19:40) A Young Lady's Illustrated Primer

(21:48) Get Involved

(21:59) Introducing

(23:01) In Other AI News

(24:43) Show Me the Money

(27:54) Common Ground

(30:46) Quiet Speculations

(40:03) 'AI Progress Is Slowing Down' Is Not Slowing Down

(42:35) Bubble, Bubble, Toil and Trouble

(44:42) The Quest for Government Money

(50:05) Chip City

(54:43) The Week in Audio

(55:46) Rhetorical Innovation

(01:02:49) Galaxy Brain Resistance

(01:11:21) Misaligned!

(01:12:20) Aligning a Smarter Than Human Intelligence is Difficult

(01:17:15) Messages From Janusworld

(01:21:19) You'll Know

(01:23:42) People Are Worried About AI Killing Everyone

(01:29:12) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Pope is a remarkably wise and helpful man. He offered us some wisdom.

Yes, he is generally playing on easy mode by saying straightforwardly true things, but that's meeting the world where it is. You have to start somewhere.

Some rejected his teachings.

Wisdom Is Offered

Two thousand years after Jesus famously got nailed to a cross for suggesting we all be nice to each other for a change, Pope Leo XIV issues a similarly wise suggestion.

Pope Leo XIV: Technological innovation can be a form of participation in the divine act of creation. It carries an ethical and spiritual weight, for every design choice expresses a vision of humanity. The Church therefore calls all builders of #AI to cultivate moral discernment as a fundamental part of their work—to develop systems that reflect justice, solidarity, and a genuine reverence for life.

The world needs honest and courageous entrepreneurs and communicators who care for the common good. We sometimes hear the saying: “Business is business!” In reality, it is not so. No one is absorbed by an organization to the point of becoming a mere cog or a simple function. Nor can there [...]

---

Outline:

(00:26) Wisdom Is Offered

(02:57) The Context of The Meme Andreessen Used (If You Don't Know)

(04:50) Andreessen Takes Bold Stand Against Moral Discernment

(07:33) Tech World Decides Performative Cruelty May Have Gone Too Far

(11:17) The Avatar of Societal Decay

(12:11) Marc's Technical Takes And Arguments Also Are Not Good

(13:12) The Best Defense

(15:24) You're All Wondering Why You're Here Today

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I previously covered Kimi K2, which now has a new thinking version. As I said at the time back in July, price in that the thinking version is coming.

Is it the real deal?

That depends on what level counts as the real deal. It's a good model, sir, by all accounts. But there have been fewer accounts than we would expect if it was a big deal, and it doesn’t fall into any of my use cases.

Introducing K2 Thinking

Kimi.ai: Hello, Kimi K2 Thinking!

The Open-Source Thinking Agent Model is here.

SOTA on HLE (44.9%) and BrowseComp (60.2%)

Executes up to 200 – 300 sequential tool calls without human interference

Excels in reasoning, agentic search, and coding

256K context window

Built as a thinking agent, K2 Thinking marks our latest efforts in test-time scaling — scaling both thinking tokens and tool-calling turns.

K2 Thinking is now live on http://kimi.com in chat mode, with full agentic mode coming soon. It is also accessible via API.

API here, Tech blog here, Weights and code here.

(Pliny jailbreak here.)

It's got 1T parameters, and Kimi and [...]

---

Outline:

(00:34) Introducing K2 Thinking

(02:15) Writing Quality

(03:07) Agentic Tool Use

(04:06) Overall

(05:08) Are Benchmarks Being Targeted?

(06:23) Just As Good Syndrome

(07:02) Reactions

(09:59) Otherwise It Has Been Strangely Quiet

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Preventing Value Drift

Maximizing Good Makes People Look Bad

---

Outline:

(00:16) Preventing Value Drift

(01:10) Maximizing Good Makes People Look Bad

(02:10) No We Have No Tuition

(03:15) Will MacAskill and the Dangers of PR Focus

(10:51) Tag, You're It

(13:16) Impact Philanthropy

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was very clearly one of those. So here we go.

As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary.

If I am quoting directly I use quote marks, otherwise assume paraphrases.

The entire conversation takes place with an understanding that no one is to mention existential risk or the fact that the world will likely transform, without stating this explicitly. Both participants are happy to operate that way. I’m happy to engage in that conversation (while pointing out its absurdity in some places), but assume that every comment I make has an implicit ‘assuming normality’ qualification on it, even when I don’t say so explicitly.

On The Sam Altman Production Function

---

Outline:

(01:10) On The Sam Altman Production Function

(02:12) On Hiring Hardware People

(04:45) On What GPT-6 Will Enable

(10:25) On government backstops for AI companies

(15:03) On monetizing AI services

(19:51) On AI's future understanding of intangibles

(24:54) On Chip-Building

(27:52) On Sam's outlook on health, alien life, and conspiracy theories

(32:16) On regulating AI agents

(34:30) On new ways to interface with AI

(36:42) On how normies will learn to use AI

(40:42) On AI's effect on the price of housing and healthcare

(44:14) On reexamining freedom of speech

(47:55) On humanity's persuadability

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.