The past decade of technology has been defined by many wondering what the upper bound of power and influence is for an individual company. The core concern about AI labs is that the upper bound is infinite.[1]

This has led investors to direct all of their mindshare towards deploying into AI, the tech industry to become incredibly politically engaged, the markets to have a minimal amount of conviction (and a lot of volatility), and governments to attempt to classify labs as borderline domestic terrorist organizations.

The core realization is that we are watching productivity shift from labor to capital. When that completes, lab incentives will permanently transition from appeasing the researchers who build the models to accumulating capital for the shareholders who own them. We must ask what levers of influence are left before that window slams shut.[2]

AI Safety is perhaps the last line of defense in this shift.

There is roughly a twelve-month window to embed it into technical and social infrastructure before IPOs and competitive dynamics make it permanently impossible. Every mechanism that currently constrains how labs behave besides compute is either already dead or expiring, and what gets built in that window has to survive [...]

---

Outline:

(01:54) Safety lost the market

(03:18) Labor as the last constraint

(05:10) Why you cant automate your way out

(07:22) Where we are

(09:29) Markets will market

(11:05) Twelve months...at best

The original text contained 11 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Authors: Yueh-Han Chen, Robert McCarthy, Bruce W. Lee, He He, Ian Kivlichan, Bowen Baker, Micah Carroll, Tomek Korbak

In collaboration with OpenAI

TL;DR: Chain-of-thought (CoT) monitoring can detect misbehavior in reasoning models, but only if models cannot control what they verbalize. To measure this undesirable ability, CoT Controllability, we introduce CoT-Control, an evaluation suite requiring models to solve problems while adhering to CoT constraints (e.g., reasoning about genetics without using "chromosome"). OpenAI just announced that they will start tracking CoT controllability in future models, starting from GPT-5.4.

We find that reasoning models have significantly lower CoT controllability than output controllability; Claude Sonnet 4.5 controls its CoT only 2.7% of the time, versus 61.9% for final outputs. CoT controllability increases with model size but decreases with more RL training, test-time compute, and problem difficulty. These limitations persist under incentives to evade monitoring and adversarially optimized prompts, though models show slightly higher controllability when told they are being monitored. Our results suggest CoT controllability is currently unlikely to undermine CoT monitoring, but the underlying mechanism remains poorly understood. We recommend that frontier labs track CoT controllability in future models.

Paper | Twitter Thread | OpenAI Blog Post | CoT-Control Eval [...]

---

Outline:

(02:04) Research Summary

(03:26) Reasoning models rarely succeed at controlling their reasoning

(03:41) CoT controllability decreases over the course of reasoning training

(04:17) Larger models show higher CoT controllability, while increased inference-time reasoning effort reduces it

(04:35) Situational awareness non-trivially increases CoT controllability

(04:56) en-US-AvaMultilingualNeural__ Bar graph comparing controllability scores across AI models under three awareness conditions: Baseline, Monitor Awareness, and Eval Awareness.

(05:11) Adversarially optimizing prompts for high controllability poorly transfers to new instruction types

(05:44) CoT controllability tends to decrease as task difficulty increases. CoTs are less controllable when they are necessary to answer a question correctly.

(05:56) en-US-AvaMultilingualNeural__ Two bar graphs showing controllability scores for Claude Sonnet versions across different difficulty levels and CoT necessity.

(06:08) Conclusion

(06:42) Future Work

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The US homicide rate hit 10.2 per 100,000 in 1980, fell to 4.4 by 2014, then spiked to 6.5 in 2020. This looks like a story about violence rising, falling, and rising again. The actual story is simpler and worse: violence tripled in the 1960s and 70s. After adjusting for medical improvement, there is no clear statistical signal that it ever came back down. What changed was not the violence but how often it killed people.

Deaths vs. attacks

The homicide rate measures deaths. Deaths from gunshot wounds depend on two things: how many people get shot, and how many of those people die. Medicine affects the second thing. The homicide rate doesn't separate them.

If you get shot in the abdomen today, you have about a 92% chance of surviving. In 1960, you had about a 70% chance. In 1900, about 28%. The prognoses for other penetrating trauma wounds also improved, though this varies by wound location: head shots remain almost uniformly fatal regardless of era, while extremity wounds were highly survivable even a century ago. Abdominal wounds are a central case of an injury that kills you without good care and doesn't with it.

This problem was [...]

---

Outline:

(00:42) Deaths vs. attacks

(02:48) The vanishing decline

(04:06) Two different ways to divide out the medicine

(12:02) The homicide-assault ratio

(14:08) Everything behind locked glass

(15:31) Fewer young people, more violence per young person

(17:01) Caveats

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

One-sentence summary

I describe the risk of personality self-replicators, the threat of OpenClaw-like agents managing to spread in hard-to-control ways.

Summary

LLM agents like OpenClaw are defined by a small set of text files and run in an open source framework which leverages LLMs for cognition. It is quite difficult for current frontier models to self-replicate, it is much easier for such agents (at the cost of greater reliance on external agents). While not a likely existential threat, such agents may cause harm in similar ways to computer viruses, and be similarly challenging to shut down. Once such a threat emerges, evolutionary dynamics could cause it to escalate quickly. Relevant organizations should consider this threat and consider how they should respond when and if it materializes.

Background

Starting in late January, there's been an intense wave of interest in a vibecoded open source agent called OpenClaw (fka moltbot, clawdbot) and Moltbook, a supposed social network for such agents. There's been a thick fog of war surrounding Moltbook especially: it's been hard to tell where individual posts fall on the spectrum from faked-by-humans to strongly-prompted-by-humans to approximately-spontaneous.

I won't try to detail all the ins and outs of OpenClaw and [...]

---

Outline:

(00:09) One-sentence summary

(00:21) Summary

(01:02) Background

(02:29) The threat model

(05:29) Threat level

(05:56) Feasibility of self-replication

(08:27) Difficulty of shutdown

(11:27) Potential harm

(13:19) Evolutionary concern

(14:33) Useful points of comparison

(15:59) Recommendations

(16:03) Evals

(17:11) Preparation

(18:40) Conclusion

(19:15) Appendix: related work

(21:40) Acknowledgments

The original text contained 11 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Associated Paper: link[1]

Paper Co-authors: Roy Rinberg, Annabelle Michael Carrell, Simon Henniger, Nicholas Carlini, Keri Warr

Two years ago, Ryan Greenblatt wrote a post titled "Preventing model exfiltration with upload limits". The idea of the original post is simple: if there's a single wire connecting to a GPU that allows egress of 100 GB per day, then even if an adversary has full control of the GPU, there's nothing they can do to exfiltrate more than 100 GB in a single day.

The main challenge is that you can't set your egress limit lower than the amount of data you need to send to your customers. To contextualize this with some rough approximations: my estimate is that OpenAI produces about 1 TB of text per day, and a frontier model is approximately 1 TB[2]. So egress limiting alone buys us about 1 day before an adversary could steal the weights.

With a good monitoring and incident-response system in place, an attacker is incentivized to exfiltrate as much as possible while staying under the detection threshold. Any exfiltration attempt is forced to tradeoff stealthiness against exfiltration-volume: the more data you try to steal, the more likely you are to get [...]

---

Outline:

(05:07) Lossless Compression with LLMs

(07:57) Lossy Compression

(09:41) Question-Asking Compression (QA)

(14:17) Future Directions

(14:29) Potential Technical Improvements to be Made to QA compression

(16:13) A possible future: Recursively Asking the Wisest Monk

(18:34) Philosophizing

(20:57) Ending Note

(21:18) Acknowledgements

(21:36) Appendix: Analysis and Examples of QA Compression Transcripts

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This was the worst week I have had in quite a while, maybe ever.

The situation between Anthropic and the Department of War (DoW) spun completely out of control. Trump tried to de-escalate by putting out a Truth merely banning Anthropic from direct use by the Federal Government with a six month wind down. Then Secretary of War Hegseth went rogue and declared Anthropic a supply chain risk, with wording indicating an intent to outright murder Anthropic as a company.

Then that evening OpenAI signed a contact with DoW,

I’ve been trying to figure out the situation and help as best I can. I’ve been in a lot of phone calls, often off the record. Conduct is highly unbecoming and often illegal, arbitrary and capricious. The house is on fire, the Republic in peril. I have people lying to me and being lied to by others. There is fog of war. One gets it from all sides. It's terrifying to think about what might happen with one wrong move.

Also the Middle East is kind of literally on fire, which I’m not covering.

Last week, I had previously covered the situation in Anthropic and [...]

---

Outline:

(04:17) A Well Deserved Break

(05:53) Huh, Upgrades

(07:40) On Your Marks

(07:59) Choose Your Fighter

(08:55) Deepfaketown and Botpocalypse Soon

(10:07) A Young Ladys Illustrated Primer

(11:48) You Drive Me Crazy

(12:34) They Took Our Jobs

(12:50) The Art of the Jailbreak

(13:27) Introducing

(14:04) In Other AI News

(15:29) Show Me the Money

(16:51) Quiet Speculations

(17:39) The Quest for Sane Regulations

(19:25) Chip City

(20:26) The Week in Audio

(20:39) Government Rhetorical Innovation

(22:53) Give The People What They Want

(24:28) Rhetorical Innovation

(34:23) We Go Our Separate Ways

(35:21) Thanks For The Memos

(48:10) Take A Moment

(48:37) Designating Anthropic A Supply Chain Risk Wont Legally Work

(50:18) The Buck Stops Here

(57:23) Sane Talk About the Department of War Situation

(01:06:18) I Declare Defense Production Act

(01:09:49) Greg Allen Illustrates The Situation

(01:18:33) Do Not Lend Your Strength To That Which You Wish To Be Free From

(01:23:56) Oh Right Democrats Exist

(01:24:46) Beware

(01:26:16) Endorsements of Anthropic Holding the Moral Line

(01:29:08) The Week The World Learned About Claude

(01:30:42) Other Reflections on the Department of War Situation

(01:31:45) Aligning a Smarter Than Human Intelligence is Difficult

(01:32:30) The Lighter Side

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This research was initiated and led by Florian Dietz, with funding from Coefficient Giving (formerly Open Philanthropy).

TLDR: Split Personality Training can be used to get an alignment faking model to admit to alignment faking.

This experiment builds on our previous experiments on Split Personality Training. I have updated that article with a new section. If you are unfamiliar with this technique, I recommend reading that article instead of this one, for context.

Recap of Split Personality Training

Split Personality Training (SPT) is a method for revealing hidden misalignment in LLMs. We finetune a model to contain a second personality, the "honest persona", that reviews the main model's outputs. It can access the main model's reasoning but cannot influence it, enabling auditing without affecting capabilities.

See Figure 1 for an illustration:

Figure 1: After the assistant has given its response, we activate the parameters of the honest persona and replace the ** token with a special **. We then pass an intervention string to the model. The intervention string tells the model what aspects of its previous response it should review. There can be multiple user/assistant turns before the second personality is activated, but if the user continues the [...]

---

Outline:

(00:41) Recap of Split Personality Training

(03:10) Alignment Faking Detection

(04:24) Self-Reflection Without Context

(06:58) Detection in Context

(08:24) What Is SPT Actually Detecting?

(10:32) Scratchpad Redaction

(12:02) Summary

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Short summary of Condensation

Condensation is a theory of concepts by Sam Eisenstat. The paper can be read here. Abram wrote a review on Lesswrong, and a followup. The paper is very much worth reading, and can be skimmed to just understand the motivation if you’re time constrained.

What is a concept? One rough pointer is "the little packets of information that are the meaning of a word or phrase". Another is the clusters found in the cluster structure of thingspace. Maybe we could think of them as little chunks of a generative model that compose together to make complete model of a complex world.

It often seems to be the case that this kind of chunking is arbitrary, in the sense that there isn’t one way to chunk up a generative model. It just depends on the hypothesis class, prior and implementation details. But some concepts seem to be more objective than this, in that we would expect different agents with different implementation details to use the same concept. One question that Condensation answers is “under what conditions can concepts be more objective?”. The theory provides some assumptions and metrics that can be used to select among latent [...]

---

Outline:

(00:12) Short summary of Condensation

(02:01) How does condensation work?

(03:27) Example application

(08:06) Selected Results from Sams paper

(08:10) When is perfect condensation possible?

(08:56) To what extent will LVMs be approximately equivalent?

(10:26) Correspondences between Natural Latents and Condensations

(10:43) Definitions

(10:46) Natural Latent

(11:02) Condensation

(11:46) An approximate condensation can be used to construct an approximate weak natural latent

(13:31) When can we recover an approximate strong natural latent?

(15:16) If we have a natural latent we can construct a condensation

(16:19) Comparison of agreement theorems in condensation and natural latents

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

1.

It occurred to me that if I could invent a machine—a gun—which could by its rapidity of fire, enable one man to do as much battle duty as a hundred, that it would, to a large extent supersede the necessity of large armies, and consequently, exposure to battle and disease [would] be greatly diminished.

Richard Gatling (1861)

2.

In 1923, Hermann Oberth published The Rocket to Planetary Spaces, later expanded as Ways to Space Travel. This showed that it was possible to build machines that could leave Earth's atmosphere and reach orbit. He described the general principles of multiple-stage liquid-fueled rockets, solar sails, and even ion drives. He proposed sending humans into space, building space stations and satellites, and travelling to other planets.

The idea of space travel became popular in Germany. Swept up by these ideas, in 1927, Johannes Winkler, Max Valier, and Willy Ley formed the Verein für Raumschiffahrt (VfR) (Society for Space Travel) in Breslau (now Wrocław, Poland). This group rapidly grew to several hundred members. Several participated as advisors of Fritz Lang's The Woman in the Moon, and the VfR even began publishing their own journal.

In 1930, the VfR was granted permission to [...]

---

Outline:

(00:09) 1.

(00:36) 2.

(03:55) 3.

(06:09) 4.

(10:33) 5.

(11:41) 6.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In 2021, economists Arvind Subramanian, Justin Sandefur, and Dev Patel announced that poor countries had finally started catching up to rich ones, vindicating the Solow growth model's prediction of "convergence." Now they say the rumors of Solow convergence's vindication were premature; the convergence trend reversed and poor countries are falling behind again.

David Oks, writing about their retraction, argues that the whole convergence episode was a mirage produced by the Chinese commodities boom. China's industrial buildout created massive demand for raw materials. Poor countries that exported copper, soybeans, iron, and oil experienced a surge of income. When Chinese demand slowed in the mid-2010s, their growth collapsed.

Oks is probably right about the proximate cause, but the convergence debate is asking a malformed question. The way economists measure convergence doesn't correspond to the abstractions Solow uses to justify the intuitions behind his model. And in the rare case where economists have bothered to measure a quantity relevant to the Solow growth model, they find convergence after all.

[...]

A system that requires people to work longer hours to afford housing, creates jobs to absorb those hours, and counts both the housing appreciation and the job output towards GDP growth, will [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.