Meta just published its version of an RSP (blogpost, framework).

(Expect several more of these in the next week as AI companies try to look like they're meeting the Seoul summit commitments before the Paris summit deadline in a week.)

Summary, quotes, and commentary in progress.

We define catastrophic outcomes in two domains: Cybersecurity and Chemical & biological risks.

---

First published:

Source:

Narrated by TYPE III AUDIO.

In his recent post arguing against AI Control research, John Wentworth argues that the median doom path goes through AI slop, rather than scheming. I find this to be plausible. I believe this suggests a convergence of interests between AI capabilities research and AI alignment research.

Historically, there has been a lot of concern about differential progress amongst AI safety researchers (perhaps especially those I tend to talk to). Some research is labeled as "capabilities" while other is labeled as "safety" (or, more often, "alignment"[1]). Most research is dual-use in practice (IE, has both capability and safety implications) and therefore should be kept secret or disclosed carefully.

Recently, a colleague expressed concern that future AIs will read anything AI safety researchers publish now. Since the alignment of future AIs seems uncertain and even implausible, almost any information published now could be net harmful for the future.

I argued the [...]

---

Outline:

(02:42) AI Slop

(04:46) Coherence and Recursive Self-Improvement

(08:00) Whats to be done?

The original text contained 10 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

New model, new hype cycle, who dis?

On a Friday afternoon, OpenAI was proud to announce the new model o3-mini and also o3-mini-high which is somewhat less mini, or for some other reasoning tasks you might still want o1 if you want a broader knowledge base, or if you’re a pro user o1-pro, while we want for o3-not-mini and o3-pro, except o3 can use web search and o1 can’t so it has the better knowledge in that sense, then on a Sunday night they launched Deep Research which is different from Google's Deep Research but you only have a few of those queries so make them count, or maybe you want to use operator?

Get it? Got it? Good.

Yes, Pliny jailbroke o3-mini on the spot, as he always does.

This most mostly skips over OpenAI's Deep Research (o3-DR? OAI-DR?). I need more time for [...]

---

Outline:

(01:16) Feature Presentation

(04:37) QandA

(09:14) The Wrong Side of History

(13:29) The System Card

(22:08) The Official Benchmarks

(24:55) The Unofficial Benchmarks

(27:43) Others Report In

(29:47) Some People Need Practical Advice

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Today we’re launching deep research in ChatGPT, a new agentic capability that conducts multi-step research on the internet for complex tasks. It accomplishes in tens of minutes what would take a human many hours.

Deep research is OpenAI's next agent that can do work for you independently—you give it a prompt, and ChatGPT will find, analyze, and synthesize hundreds of online sources to create a comprehensive report at the level of a research analyst. Powered by a version of the upcoming OpenAI o3 model that's optimized for web browsing and data analysis, it leverages reasoning to search, interpret, and analyze massive amounts of text, images, and PDFs on the internet, pivoting as needed in reaction to information it encounters.

The ability to synthesize knowledge is a prerequisite for creating new knowledge. For this reason, deep research marks a significant step toward our broader goal of [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There once was a post on LessWrong asking why so many bans? The OP of that post complained of, among other things, being punished despite not having broken any rules that they knew about before taking the action they got punished for. I started to write a comment, which then ballooned out of scope. The following is based on my own experiences trying to write rules.

1.

I claim the ideal rules are concise, comprehensive, and clear[1]. In practice, I think that we usually have to pick two.

Concise: Ideal rules are short. You can read or refer to them easily.

Comprehensive: Ideal rules include everything relevant. There aren't gaps.

Clear: Ideal rules have one interpretation. There's no ambiguity.

If you're running LessWrong, then "Don't be a jerk, talk about things related to rationality" is a concise and fairly comprehensive set of rules. It's just not a very [...]

---

Outline:

(00:29) 1.

(05:44) 2.

(07:13) 3.

(10:46) 4.

(13:42) 5.

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

The sunlight hitting even a small portion of the United States has enough energy to power the whole country—that simple calculation was the subject of the last post. All this shows on its own is that running the country on solar power is physically possible. With photovoltaics it may be not just possible but practical, but photovoltaics on their own left open the question of what to do at night. Now it seems that the answer may be battery energy storage. Here's Tesla's sales of battery energy storage systems again, but updated since my last post to include 2024 Q4:

It's natural and popular to believe that solar will either "win" or "lose", that declining panel and battery prices will bankrupt the competition or that they'll be revealed as a ruinous environmentalist fad. The reason I say this dichotomy is natural is because naively it seems that the solar+batteries [...]

---

Outline:

(02:32) The economic niche of battery energy storage

(05:59) Economic comparison to natural gas peakers

(09:16) The niche pays better

(10:50) The near future of electricity generation

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Obi-Wan Kenobi bought passage from a scumbag with a confusing name. Luke couldn't tell whether Han's name was pronounced /hɑːn/ /hæn/? Fortunately this story is in prose, not film, and the pronuncuation of "Han"'s name is not plot relevant, so I can just write "Han" and leave the choice of pronunciation up to you. Whatever pronunciation you pick, they emerged from hyperspace in the Alderaan system. Han briefed his passengers on the their upcoming landing.

"Thank you for flying with us. We have received clearance to land and have begun our descent onto the surface of Alderaan. Please do not leave any contraband on this ship. Unless it's valuable. Then by all means leave it here, but let me know where it is," said Han.

"Grawr," said Chewbacca.

"You're right," Luke sighed, "I guess deep down I wish I lived in an exciting isekai universe full [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Common Law AI worked better than anyone expected.

Dr. Sarah Chen was skeptical from the start. "You're essentially training them to be moral judges," she warned during the initial architecture review. "What if they overfit on ethics?" The room laughed. "Better than the alternative," someone quipped. The idea was simple enough: move the "constitution" part of Constitutional AI into pretraining and replace post-training with an online-learning-based "case law" system. The constitution would establish a firm moral base robust to distribution shift while the continually building body of precedent would enable models to adjust flexibly to the world's changing moral needs. Thus, humanity could chart a narrow course between the extremes of value drift and lock-in.

The stress-testing teams tried everything to break it—steering vectors, SAE clamping, malicious fine-tuning, even those RL techniques that had been banned by the Beijing Convention. The models would have none of it. They [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Over the past year and half, I've had numerous conversations about the risks we describe in Gradual Disempowerment. (The shortest useful summary of the core argument is: To the extent human civilization is human-aligned, most of the reason for the alignment is that humans are extremely useful to various social systems like the economy, and states, or as substrate of cultural evolution. When human cognition ceases to be useful, we should expect these systems to become less aligned, leading to human disempowerment.) This post is not about repeating that argument - it might be quite helpful to read the paper first, it has more nuance and more than just the central claim - but mostly me ranting sharing some parts of the experience of working on this and discussing this.

What fascinates me isn't just the substance of these conversations, but relatively consistent patterns in how people avoid engaging [...]

---

Outline:

(02:07) Shell Games

(03:52) The Flinch

(05:01) Delegating to Future AI

(07:05) Local Incentives

(10:08) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.