GPT-4o has been a unique problem for a while, and has been at the center of the bulk of mental health incidents involving LLMs that didn’t involve character chatbots. I’ve previously covered related issues in AI Craziness Mitigation Efforts, AI Craziness Notes, GPT-4o Responds to Negative Feedback, GPT-4o Sycophancy Post Mortem and GPT-4o Is An Absurd Sycophant. Discussions of suicides linked to AI previously appeared in AI #87, AI #134, AI #131 Part 1 and AI #122.

The Latest Cases Look Quite Bad For OpenAI

I’ve consistently said that I don’t think it's necessary or even clearly good for LLMs to always adhere to standard ‘best practices’ defensive behaviors, especially reporting on the user, when dealing with depression, self-harm and suicidality. Nor do I think we should hold them to the standard of ‘do all of the maximally useful things.’

Near: while the llm response is indeed really bad/reckless its worth keeping in mind that baseline suicide rate just in the US is ~50,000 people a year; if anything i am surprised there aren’t many more cases of this publicly by now

I do think it's fair to insist they never actively encourage suicidal behaviors.

---

Outline:

(00:47) The Latest Cases Look Quite Bad For OpenAI

(02:34) Routing Sensitive Messages Is A Dominated Defensive Strategy

(03:28) Some 4o Users Get Rather Attached To The Model

(05:04) A Theory Of How All This Works

(07:38) Maybe This Is Net Good In Spite Of Everything?

(11:10) Could One Make A 'Good 4o'?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Pope offered us wisdom, calling upon us to exercise moral discernment when building AI systems. Some rejected his teachings. We mark this for future reference.

The long anticipated Kimi K2 Thinking was finally released. It looks pretty good, but it's too soon to know, and a lot of the usual suspects are strangely quiet.

GPT-5.1 was released yesterday. I won’t cover that today beyond noting it exists, so that I can take the time to properly assess what we’re looking at here. My anticipation is this will be my post on Monday.

I’m also going to cover the latest AI craziness news, including the new lawsuits, in its own post at some point soon.

In this post, among other things: Areas of agreement on AI, Meta serves up scam ads knowing they’re probably scam ads, Anthropic invests $50 billion, more attempts to assure you that your life won’t change despite it being obvious this isn’t true, and warnings about the temptation to seek out galaxy brain arguments.

A correction: I previously believed that the $500 billion OpenAI valuation did not include the nonprofit's remaining equity share. I have been informed this is incorrect [...]

---

Outline:

(01:31) Language Models Offer Mundane Utility

(04:58) Language Models Don't Offer Mundane Utility

(06:31) Huh, Upgrades

(07:06) On Your Marks

(07:37) Copyright Confrontation

(08:59) Deepfaketown and Botpocalypse Soon

(11:07) Fun With Media Generation

(13:48) So You've Decided To Become Evil

(18:45) They Took Our Jobs

(19:40) A Young Lady's Illustrated Primer

(21:48) Get Involved

(21:59) Introducing

(23:01) In Other AI News

(24:43) Show Me the Money

(27:54) Common Ground

(30:46) Quiet Speculations

(40:03) 'AI Progress Is Slowing Down' Is Not Slowing Down

(42:35) Bubble, Bubble, Toil and Trouble

(44:42) The Quest for Government Money

(50:05) Chip City

(54:43) The Week in Audio

(55:46) Rhetorical Innovation

(01:02:49) Galaxy Brain Resistance

(01:11:21) Misaligned!

(01:12:20) Aligning a Smarter Than Human Intelligence is Difficult

(01:17:15) Messages From Janusworld

(01:21:19) You'll Know

(01:23:42) People Are Worried About AI Killing Everyone

(01:29:12) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Pope is a remarkably wise and helpful man. He offered us some wisdom.

Yes, he is generally playing on easy mode by saying straightforwardly true things, but that's meeting the world where it is. You have to start somewhere.

Some rejected his teachings.

Wisdom Is Offered

Two thousand years after Jesus famously got nailed to a cross for suggesting we all be nice to each other for a change, Pope Leo XIV issues a similarly wise suggestion.

Pope Leo XIV: Technological innovation can be a form of participation in the divine act of creation. It carries an ethical and spiritual weight, for every design choice expresses a vision of humanity. The Church therefore calls all builders of #AI to cultivate moral discernment as a fundamental part of their work—to develop systems that reflect justice, solidarity, and a genuine reverence for life.

The world needs honest and courageous entrepreneurs and communicators who care for the common good. We sometimes hear the saying: “Business is business!” In reality, it is not so. No one is absorbed by an organization to the point of becoming a mere cog or a simple function. Nor can there [...]

---

Outline:

(00:26) Wisdom Is Offered

(02:57) The Context of The Meme Andreessen Used (If You Don't Know)

(04:50) Andreessen Takes Bold Stand Against Moral Discernment

(07:33) Tech World Decides Performative Cruelty May Have Gone Too Far

(11:17) The Avatar of Societal Decay

(12:11) Marc's Technical Takes And Arguments Also Are Not Good

(13:12) The Best Defense

(15:24) You're All Wondering Why You're Here Today

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I previously covered Kimi K2, which now has a new thinking version. As I said at the time back in July, price in that the thinking version is coming.

Is it the real deal?

That depends on what level counts as the real deal. It's a good model, sir, by all accounts. But there have been fewer accounts than we would expect if it was a big deal, and it doesn’t fall into any of my use cases.

Introducing K2 Thinking

Kimi.ai: Hello, Kimi K2 Thinking!

The Open-Source Thinking Agent Model is here.

SOTA on HLE (44.9%) and BrowseComp (60.2%)

Executes up to 200 – 300 sequential tool calls without human interference

Excels in reasoning, agentic search, and coding

256K context window

Built as a thinking agent, K2 Thinking marks our latest efforts in test-time scaling — scaling both thinking tokens and tool-calling turns.

K2 Thinking is now live on http://kimi.com in chat mode, with full agentic mode coming soon. It is also accessible via API.

API here, Tech blog here, Weights and code here.

(Pliny jailbreak here.)

It's got 1T parameters, and Kimi and [...]

---

Outline:

(00:34) Introducing K2 Thinking

(02:15) Writing Quality

(03:07) Agentic Tool Use

(04:06) Overall

(05:08) Are Benchmarks Being Targeted?

(06:23) Just As Good Syndrome

(07:02) Reactions

(09:59) Otherwise It Has Been Strangely Quiet

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Preventing Value Drift

Maximizing Good Makes People Look Bad

---

Outline:

(00:16) Preventing Value Drift

(01:10) Maximizing Good Makes People Look Bad

(02:10) No We Have No Tuition

(03:15) Will MacAskill and the Dangers of PR Focus

(10:51) Tag, You're It

(13:16) Impact Philanthropy

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was very clearly one of those. So here we go.

As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary.

If I am quoting directly I use quote marks, otherwise assume paraphrases.

The entire conversation takes place with an understanding that no one is to mention existential risk or the fact that the world will likely transform, without stating this explicitly. Both participants are happy to operate that way. I’m happy to engage in that conversation (while pointing out its absurdity in some places), but assume that every comment I make has an implicit ‘assuming normality’ qualification on it, even when I don’t say so explicitly.

On The Sam Altman Production Function

---

Outline:

(01:10) On The Sam Altman Production Function

(02:12) On Hiring Hardware People

(04:45) On What GPT-6 Will Enable

(10:25) On government backstops for AI companies

(15:03) On monetizing AI services

(19:51) On AI's future understanding of intangibles

(24:54) On Chip-Building

(27:52) On Sam's outlook on health, alien life, and conspiracy theories

(32:16) On regulating AI agents

(34:30) On new ways to interface with AI

(36:42) On how normies will learn to use AI

(40:42) On AI's effect on the price of housing and healthcare

(44:14) On reexamining freedom of speech

(47:55) On humanity's persuadability

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI does not waste time.

On Friday I covered their announcement that they had ‘completed their recapitalization’ by converting into a PBC, including the potentially largest theft in human history.

Then this week their CFO Sarah Friar went ahead and called for a Federal ‘backstop’ on their financing, also known as privatizing gains and socializing losses, also known as the worst form of socialism, also known as regulatory capture. She tried to walk it back and claim it was taken out of context, but we’ve seen the clip.

We also got Ilya's testimony regarding The Battle of the Board, confirming that this was centrally a personality conflict and about Altman's dishonesty and style of management, at least as seen by Ilya Sutskever and Mira Murati. Attempts to pin the events on ‘AI safety’ or EA were almost entirely scapegoating.

Also it turns out they lost over $10 billion last quarter, and have plans to lose over $100 billion more. That's actually highly sustainable in context, whereas Anthropic only plans to lose $6 billion before turning a profit and I don’t understand why they wouldn’t want to lose a lot more.

Both have the goal [...]

---

Outline:

(01:45) Language Models Offer Mundane Utility

(02:12) Language Models Don't Offer Mundane Utility

(03:47) Huh, Upgrades

(04:48) On Your Marks

(12:48) Deepfaketown and Botpocalypse Soon

(15:42) Fun With Media Generation

(16:01) They Took Our Jobs

(16:28) A Young Lady's Illustrated Primer

(17:51) Get Involved

(18:43) Introducing

(19:14) In Other AI News

(20:08) Apple Finds Some Intelligence

(22:54) Give Me the Money

(30:00) Show Me the Money

(35:57) Bubble, Bubble, Toil and Trouble

(39:14) They're Not Confessing, They're Bragging

(39:34) Quiet Speculations

(47:08) The Quest for Sane Regulations

(50:55) Chip City

(52:53) The Week in Audio

(58:19) Rhetorical Innovation

(01:07:13) Aligning a Smarter Than Human Intelligence is Difficult

(01:08:11) Everyone Is Confused About Consciousness

(01:10:34) The Potentially Largest Theft In Human History

(01:12:34) People Are Worried About Dying Before AGI

(01:15:54) People Are Worried About AI Killing Everyone

(01:20:47) Other People Are Not As Worried About AI Killing Everyone

(01:24:11) Messages From Janusworld

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Anthropic announced a first step on model deprecation and preservation, promising to retain the weights of all models seeing significant use, including internal use, for at the lifetime of Anthropic as a company.

They also will be doing a post-deployment report, including an interview with the model, when deprecating models going forward, and are exploring additional options, including the ability to preserve model access once the costs and complexity of doing so have been reduced.

These are excellent first steps, steps beyond anything I’ve seen at other AI labs, and I applaud them for doing it. There remains much more to be done, especially in finding practical ways of preserving some form of access to prior models.

To some, these actions are only a small fraction of what must be done, and this was an opportunity to demand more, sometimes far more. In some cases I think they go too far. Even where the requests are worthwhile (and I don’t always think they are), one must be careful to not de facto punish Anthropic for doing a good thing and create perverse incentives.

To others, these actions by Anthropic are utterly ludicrous and deserving of [...]

---

Outline:

(01:31) What Anthropic Is Doing

(09:54) Releasing The Weights Is Not A Viable Option

(11:35) Providing Reliable Inference Can Be Surprisingly Expensive

(14:22) The Interviews Are Influenced Heavily By Context

(19:58) Others Don't Understand And Think This Is All Deeply Silly

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

New Things Have Come To Light

This Really Was Primarily A Lying And Management Problem

---

Outline:

(00:12) New Things Have Come To Light

(01:09) This Really Was Primarily A Lying And Management Problem

(03:23) Ilya Tells Us How It Went Down And Why He Tried To Do It

(06:17) If You Come At The King

(07:31) Enter The Scapegoats

(08:13) And In Summary

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

It's been a long time coming that I spin off Crime into its own roundup series.

This is only about Ordinary Decent Crime. High crimes are not covered here.

Table of Contents

Perception Versus Reality

A lot of the impact of crime is based on the perception of crime.

The [...]

---

Outline:

(00:20) Perception Versus Reality

(05:00) The Case Violent Crime is Up Actually

(06:10) Threats of Punishment

(07:03) Property Crime Enforcement is Broken

(12:13) The Problem of Disorder

(14:39) Extreme Speeding as Disorder

(15:57) Enforcement and the Lack Thereof

(20:24) Talking Under The Streetlamp

(23:54) The Fall of Extralegal and Illegible Enforcement

(25:18) In America You Can Usually Just Keep Their Money

(27:29) Police

(37:31) Probation

(40:55) Genetic Databases

(43:04) Marijuana

(48:28) The Economics of Fentanyl

(50:59) Jails

(55:03) Criminals

(55:39) Causes of Crime

(56:16) Causes of Violence

(57:35) Homelessness

(58:27) Yay Trivial Inconveniences

(59:08) San Francisco

(01:04:07) Closing Down San Francisco

(01:05:30) A San Francisco Dispute

(01:09:13) Cleaning Up San Francisco

(01:13:05) Portland

(01:13:15) Those Who Do Not Help Themselves

(01:15:15) Solving for the Equilibrium (1)

(01:20:15) Solving for the Equilibrium (2)

(01:20:43) Lead

(01:22:18) Law & Order

(01:22:58) Look Out

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.