I've met a few Landians over the last couple years, and they generally recommend that I start with reading Land's (now defunct) Xenosystems blog, or Xenosystems, a Passage Publishing book that compiles posts from the blog. While I've read some of Fanged Noumena in the past, I would agree with these Landians that Xenosystems (and currently, the book version) is the best starting point. In the current environment, where academia has lost much of its intellectual relevance, it seems overly pretentious to start with something as academic as Fanged Noumena. I mainly write in the blogosphere rather than academia, and so Xenosystems seems appropriate to review.

The book's organization is rather haphazard (as might be expected from a blog compilation). It's not chronological, but rather separated into thematic chapters. I don't find the chapter organization particularly intuitive; for example, politics appears throughout, rather than being its own chapter [...]

---

Outline:

(06:35) Neoreactionary background

(15:44) The Trichotomy

(24:16) Exiting reaction

(36:44) Outside Metaphysics

(46:19) Orthogonality versus Will-to-Think

(55:54) Meta-Neocameral Singleton?

(01:01:15) Oh my Gnon, am I going to become an AI accelerationist?

(01:12:45) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

MIRI updates

• Aaron Scher and Joe Collman have joined the Technical Governance Team at MIRI as researchers. Aaron previously did independent research related to sycophancy in language models and mechanistic interpretability, while Joe previously did independent research related to AI safety via debate and contributed to field-building work at MATS and BlueDot Impact.

• In an interview with PBS News Hour's Paul Solman, Eliezer Yudkowsky briefly explains why he expects smarter-than-human AI to cause human extinction.

• In an interview with The Atlantic's Ross Andersen, Eliezer discusses the reckless behavior of the leading AI companies, and the urgent need to change course.

News and links

• Google DeepMind announced a hybrid AI system capable of solving International Mathematical Olympiad problems at the silver medalist level. In the wake of this development, a Manifold prediction market significantly increased its odds that AI will [...]

---

Outline:

(00:07) MIRI updates

(00:51) News and links

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 108 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

TL;DR: We introduce the first comprehensive theoretical framework for understanding and mitigating secret collusion among advanced AI agents, along with CASE, a novel model evaluation framework. CASE assesses the cryptographic and steganographic capabilities of agents, while exploring the emergence of secret collusion in real-world-like multi-agent settings. Whereas current AI models aren't yet proficient in advanced steganography, our findings show rapid improvements in individual and collective model capabilities, posing unprecedented safety and security risks. These results highlight urgent challenges for AI governance and policy, urging institutions such as the EU AI Office and AI safety bodies in the UK and US to prioritize cryptographic and steganographic evaluations of frontier models. Our research also opens up critical new pathways for [...]

---

Outline:

(03:00) Multi-Agent Safety and Security in the Age of Autonomous Internet Agents

(11:00) DecAI: Decentralized Systems of AI Agents

(16:18) Steganography

(18:21) How detectable is undetectable?

(19:31) Provably Secure Steganography (aka Complexity-theoretic security)

(19:57) Information-Theoretic Security and Perfectly Secure Steganography

(21:33) Perfectly Secure Steganography via Minimum Entropy Coupling

(23:35) Collusion and Secret Collusion

(24:14) The Incentives Required for (Secret) Collusion

(26:15) On the Emergence of Secret Collusion: Instrumental vs. Intentional

(26:35) Case 1: Agents with pre-existing steganographic capabilities

(28:47) Case 2: Emergence through pre-deployment optimisation pressure

(29:29) Case 3: Emergence during deployment

(31:35) Putting it all together - understanding and evaluating existing models

(32:48) Disappearing Act: What capabilities do LLMs need to collude secretly?

(33:40) Benchmarking Frontier and Baseline Models

(36:01) Instrumental Emergence

(39:46) Mitigating Secret Collusion

(40:02) Monitoring and Penalisation

(42:50) Limiting Capabilities

(46:38) Paraphrasing and the Performance/Mitigation Trade-off

(51:58) On Intrinsic Steganographic and Cryptographic Frontier Model Capabilities

(57:29) Steganography and AI Control

(01:00:00) Policy Considerations and Impact

(01:02:20) Conclusions: Looking Ahead

(01:03:57) Open research questions

(01:06:17) A Call to Action

The original text contained 5 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This work was funded by Polaris Ventures

There is currently no consensus on how difficult the AI alignment problem is. We have yet to encounter any real-world, in the wild instances of the most concerning threat models, like deceptive misalignment. However, there are compelling theoretical arguments which suggest these failures will arise eventually.

Will current alignment methods accidentally train deceptive, power-seeking AIs that appear aligned, or not? We must make decisions about which techniques to avoid and which are safe despite not having a clear answer to this question.

To this end, a year ago, we introduced the AI alignment difficulty scale, a framework for understanding the increasing challenges of aligning artificial intelligence systems with human values.

This follow-up article revisits our original scale, exploring how our understanding of alignment difficulty has evolved and what new insights we've gained. This article will explore three main themes that have emerged [...]

---

Outline:

(02:17) The Scale

(03:17) Easy, Medium and Hard Difficulty

(03:44) Levels 1-3

(07:21) Levels 4-7

(14:47) Levels 8-10

(17:23) Dynamics of the Scale

(22:23) Increasing Costs and Challenges

(24:21) Key Factors Changing Across the Scale

(25:49) Defining Alignment Difficulty

(27:38) High Impact Tasks

(29:54) Task Difficulty and Complexity of Feedback

(34:45) Influence of Architecture

(36:19) Conclusion

The original text contained 4 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Posting something about a current issue that I think many people here would be interested in. See also the related EA Forum post.

California Governor Gavin Newsom has until September 30 to decide the fate of SB 1047 - one of the most hotly debated AI bills in the world. The Center for AI Safety Action Fund, where I work, is a co-sponsor of the bill. I’d like to share how you can help support the bill if you want to.

About SB 1047 and why it is important

SB 1047 is an AI bill in the state of California. SB 1047 would require the developers of the largest AI models, costing over $100 million to train, to test the models for the potential to cause or enable severe harm, such as cyberattacks on critical infrastructure or the creation of biological weapons resulting in mass casualties or $500 million [...]

---

Outline:

(00:34) About SB 1047 and why it is important

(02:55) How you can help

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is going to probably be a long post, so do try to get a drink and a snack while reading this post.

This is an edited version of my own comment on the post below, and I formatted and edited the content in line with what @MondSemmel recommended:

My comment: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/?commentId=Gcigdmuje4EacwirD

MondSemmel's comment: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/?commentId=WcKi4RcjRstoFFvbf

The post I'm responding to: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/

To start out my disagreement, I have this to talk about:

Response to Lethality 3

We need to get alignment right on the 'first critical try' at operating at a 'dangerous' level of intelligence, where unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don't get to try again.

I think this is actually wrong, because of synthetic data letting us control what the AI learns and what they value, and in particular we can place honeypots that are practically indistinguishable [...]

---

Outline:

(00:33) Response to Lethality 3

(03:33) Response to Lethality 6

(04:33) Response to AI fragility claims

(08:35) Response to Lethality 10

(13:08) Response to Lethality 11

(15:13) Response to Lethality 15

(16:51) Response to Lethality 16

(17:23) Response to Lethality 17

(17:55) Responses to Lethalities 18 and 19

(19:01) Response to Lethality 21

(20:57) Response to Lethality 22

(22:47) Response to Lethality 23

(23:40) Response to Lethality 24

(26:35) Response to Lethality 25

(27:05) Responses to Lethalities 28 and 29

(28:02) Response to Lethality 32

(31:42) Response to Lethality 39

(32:31) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

I just left a comment on PIBBSS' Manfund grant request (which I funded $25k) that people might find interesting. PIBBSS needs more funding!

Main points in favor of this grant

1. My inside view is that PIBBSS mainly supports “blue sky” or “basic” research, some of which has a low chance of paying off, but might be critical in “worst case” alignment scenarios (e.g., where “alignment MVPs” don’t work, or “sharp left turns” and “intelligence explosions” are more likely than I expect). In contrast, of the technical research MATS supports, about half is basic research (e.g., interpretability, evals, agent foundations) and half is applied research (e.g., oversight + control, value alignment). I think the MATS portfolio is a better holistic strategy for furthering AI safety and reducing AI catastrophic risk. However, if one takes into account the research conducted at AI labs and supported by MATS, PIBBSS’ strategy makes [...]

---

Outline:

(00:03) I just left a comment on PIBBSS Manfund grant request (which I funded $25k) that people might find interesting. PIBBSS needs more funding!

(00:18) Main points in favor of this grant

(04:21) Donors main reservations

(05:44) Process for deciding amount

(06:15) Conflicts of interest

---

First published:

Source:

Narrated by TYPE III AUDIO.

I’ve seen a fair amount of skepticism about the “Provably Safe AI” paradigm, but I think detractors give it too little credit. I suspect this is largely because of idea inoculation - people have heard an undeveloped or weak man version of the idea, for example, that we can use formal methods to state our goals and prove that an AI will do that, and have already dismissed it. (Not to pick on him at all, but see my question for Scott Aaronson here.)

I will not argue that Guaranteed Safe AI solves AI safety generally, or that it could do so - I will leave that to others. Instead, I want to provide a concrete example of a near-term application, to respond to critics who say that proveability isn’t useful because it can’t be feasibly used in real world cases when it involves the physical world, and when [...]

---

Outline:

(01:14) A path to provably safe autonomous vehicles

(02:37) Proven Input Reliability

(04:21) Proven Safe Driving

(06:39) Provably Safe ML

(08:02) Adversarial Concerns

(10:04) Defining Broader Sociotechnological Proveably Safe AI

(11:29) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

In the fields of user experience and accessibility, everyone talks about the curb cut effect: Features that are added as accommodations for people with disabilities sometimes become widely useful and beloved. But not every accommodation becomes a "curb cut," and I've been thinking about other patterns that come up when accommodations intersect with wider society.

[... Summary from the end of the post]

First, there's a trio of scenarios in which an accommodation proves helpful to more people than it was originally intended for:

• The Curb Cut Effect: When something designed as an accommodation for a particular group becomes widely useful
• The Weighted Blanket Effect: When a curb cut effect also leads to innovation or broader availability, creating additional benefits to the original users
• The Orange-in-a-Box Effect: When a curb cut effect has potential externalities or societal costs in aggregate. It requires [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

There's a popular story that goes like this: Christopher Hitchens used to be in favor of the US waterboarding terrorists because he though it's wasn't bad enough to be torture.. Then he had it tried on himself, and changed his mind, coming to believe it isn't torture.

(Context for those unfamiliar: in the decade following 9/11, the US engaged in a lot of... questionable behavior to persecute the war on terror, and there was a big debate on whether waterboarding should be permitted. Many other public figures also volunteered to undergo the procedure as a part of this public debate; most notably Sean Hannity, who was an outspoken proponent of waterboarding, yet welched on his offer and never tried it himself.)

This story intrigued me because it's popular among both Hitchens' fans and his detractors. His fans use it as an example of his intellectual honesty and willingness to [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.