Today, Retired U.S. Army General Paul M. Nakasone has joined our Board of Directors. A leading expert in cybersecurity, Nakasone's appointment reflects OpenAI's commitment to safety and security, and underscores the growing significance of cybersecurity as the impact of AI technology continues to grow.

As a first priority, Nakasone will join the Board's Safety and Security Committee, which is responsible for making recommendations to the full Board on critical safety and security decisions for all OpenAI projects and operations.

Whether this was influenced by Aschenbrenner's situational-awareness.ai or not, it's welcome to see OpenAI emphasizing the importance of security. Even with this appointment it's unclear how deeply OpenAI leadership / staff have internalized a security mindset.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Research Status: Written & researched quickly. I think the key point is fairly simple and obvious. I relied on Claude to help with rewriting.

There's been a growing interest in predicting when various products or jobs will be "fully automated." Will we soon have popular movies, books, or even CEOs that are entirely AI-generated?

Some very quickly-found links:
- https://manifold.markets/ChaseStevens/will-an-aigenerated-paper-be-accept
- https://manifold.markets/GabeGarboden/will-aigenerated-art-win-a-major-tr
- https://manifold.markets/probajoelistic/-by-2026-will-any-fully-aigenerated
- https://www.forbes.com/sites/sherzododilov/2024/01/11/can-ai-become-your-next-ceo/

It's an intriguing question, but I believe its definition is slippier than some realize. Here's why.

First, the idea of "full automation" on complex tasks (movies, books, CEO duties) is somewhat of a false dichotomy. In practice, automation exists on a spectrum, with diminishing returns at the extreme. Consider self-driving cars. Even the most advanced autonomous vehicles still have remote human monitors who can intervene if necessary. This human involvement might be significant at first (one intervention for every few miles, in the [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

We have written a paper on sandbagging for which we present the abstract and brief results in this post. See the paper for more details. Tweet thread here.

Trustworthy capability evaluations are crucial for ensuring the safety of AI systems, and are becoming a key component of AI regulation. However, the developers of an AI system, or the AI system itself, may have incentives for evaluations to understate the AI's actual capability. These conflicting interests lead to the problem of sandbagging – which we define as strategic underperformance on an evaluation. In this paper we assess [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

microwave drilling startups

I've seen a bunch of articles about startups trying to do microwave drilling of rock for geothermal energy. Multiple people have asked me about Quaise Energy. (Here's a popular video.) I'm tired of hearing about them, so I'm writing this post to explain some of the reasons why their idea is impractical.

vaporized rock condenses

When rock is vaporized, that rock vapor doesn't just disappear. What happens to it? The answer is, it would quickly condense on the hole wall and pipe.

Initially, a lot of people working on microwave drilling didn't even think about that. Once they did, they decided the solution was to use compressed air to condense the rock and blow the rock particles out. But as anyone familiar with drilling would know, that introduces new problems.

air pressure

Current drilling sometimes uses [...]

---

Outline:

(00:07) microwave drilling startups

(00:29) vaporized rock condenses

(01:01) air pressure

(02:31) energy payback

(02:35) energy usage

(03:12) thermal conductivity

(04:02) payback period

(04:40) some other problems

(04:44) waveguide losses

(05:12) reverse heat transfer

(05:32) hole collapse

(05:48) one thing at a time

(06:29) solar power exists

(06:46) this is only an example

---

First published:

Source:

Narrated by TYPE III AUDIO.

This payout report covers the Long-Term Future Fund's grantmaking from May 1 2023 to March 31 2024 (11 months). It follows our previous April 2023 payout report.

• Total funding recommended: $6,290,550
• Total funding paid out: $5,363,105
• Number of grants paid out: 141
• Acceptance rate (excluding desk rejections): 159/672 = 23.7%
• Acceptance rate (including desk rejections): 159/825 = 19.3%
• Report authors: Linchuan Zhang (primary author), Caleb Parikh (fund chair), Oliver Habryka, Lawrence Chan, Clara Collier, Daniel Eth, Lauro Langosco, Thomas Larsen, Eli Lifland

25 of our grantees, who received a total of $790,251, requested that our public reports for their grants are anonymized (the table below includes those grants). 13 grantees, who received a total of $529, 819, requested that we not include public reports for their grants. You can read our policy on public reporting here.

We referred at least 2 grants to other funders for evaluation.

Highlighted Grants

(The [...]

---

Outline:

(01:44) Highlighted Grants

(20:26) Other updates

(22:08) Other writings

(26:50) Appendix

(26:53) Other Grants We Made During This Time Period

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Apple was for a while rumored to be planning launch for iPhone of AI assisted emails, texts, summaries and so on including via Siri, to be announced at WWDC 24.

It's happening. Apple's keynote announced the anticipated partnership with OpenAI.

The bottom line is that this is Siri as the AI assistant with full access to everything on your phone, with relatively strong privacy protections. Mostly it is done on device, the rest via ‘private cloud compute.’ The catch is that when they need the best they call out for OpenAI, but they do check with you first explicitly each time, OpenAI promises not to retain data and they hide your queries, unless you choose to link up your account.

If the new AI is good enough and safe enough then this is pretty great. If Google doesn’t get its act together reasonably soon to deliver [...]

---

Outline:

(01:30) AiPhone

(04:00) Privacy

(05:27) Practical Magic

(07:38) Dance With the Devil

(08:30) Does It Work?

(10:54) Do You Dare?

(11:59) Who Pays Who?

(13:37) AiPhone Fans

(19:36) No AiPhone

(23:54) In Other Apple News

---

First published:

Source:

Narrated by TYPE III AUDIO.

Yesterday I obtained Anthropic's[1] Certificate of Incorporation, and its past versions, from the State of Delaware. I don't recommend reading it.[2] This post is about what the CoI tells us about Anthropic's Long-Term Benefit Trust (context: Maybe Anthropic's Long-Term Benefit Trust is powerless).

Tl;dr: the only new information of moderate importance is the voting thresholds necessary to modify Trust stuff. My concerns all still stand in some form. Absence of badness is a small positive update.

Anthropic has vaguely described stockholders' power over the Trust:

a series of "failsafe" provisions . . . allow changes to the Trust and its powers without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree. The required supermajorities increase as the Trust's power phases in

The CoI has details: amending the CoI to modify the Trust requires a vote reaching the "Transfer Approval Threshold," defined as:

(1) prior to [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

tl;dr

LessWrong now has a Subscribed tab (next to the Latest tab and Enriched tab[1]). You can now "follow" users, which means their posts and comments will show up in your Subscribed tab.

You can follow users via multiple methods:

1. Using the widget on the Subscribed tab:

2. You can follow people from their user profile:

3. You can follow people using the user tooltip that comes up when you hover on their username.

The full brief

A while back...notifiy on all comments. I wanted a feed.

Feed has two main advantages:

1. Can easily start to read content of post before clicking.
2. Puts comments on even footing as posts.

• This is a new "subscription type". We migrated existing to make it easier. Assume you want. Can unsubscribe
   
• High SNR vs too insular.
  • showing responses helpful

• Open problem:
  • how [...]

---

Outline:

(00:03) tl;dr

(00:43) The full brief

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

As we approach and pass through an AI takeoff period, the risk of nuclear war (or other all-out global conflict) will increase.

An AI takeoff would involve the automation of scientific and technological research. This would lead to much faster technological progress, including military technologies. In such a rapidly changing world, some of the circumstances which underpin the current peaceful equilibrium will dissolve or change. There are then two risks[1]:

1. Fundamental instability. New circumstances could give a situation where there is no peaceful equilibrium it is in everyone's interests to maintain.
   • e.g. —
     • If nuclear calculus changes to make second strike capabilities infeasible
     • If one party is racing ahead with technological progress and will soon trivially outmatch the rest of the world, without any way to credibly commit not to completely disempower them after it has done so
2. Failure [...]

---

Outline:

(02:41) Why do(n’t) people go to war?

(03:34) Rational reasons to go to war

(05:44) Irrational reasons to go to war

(06:21) Impacts of AI takeoff on reasons to go to war

(06:56) Impacts on rational reasons for war

(07:00) Commitment issues

(08:18) Private information

(09:05) Issue indivisibility

(09:59) Impacts on irrational reasons for war

(10:04) Irrational decision-making

(11:29) Misaligned decision-makers

(12:12) National pride

(13:01) Strategies for reducing risk of war

(13:06) Strategies for averting failure to navigate takeoff

(13:25) Research and dissemination

(14:02) Spreading “we’re all in this together” frames

(14:50) Agreements/treaties about sharing power of AI

(16:00) Differential technological development

(18:46) What about an AI pause?

(19:36) Closing thoughts

(19:40) What about non-nuclear warfare?

(19:56) How big a deal is this?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I want to develop rationality training, which is aimed at solving confusing problems.

Two key problems with "confusing problems" are:

1. You might feel so confused and overwhelmed that you bounce off completely.
2. You might be confused about what counts as progress, or where the most progress is possible, and accidentally work on the wrong thing.

A skill that helps with both of these is "metastrategic brainstorming" – the art of generating lots of potential good approaches, and then choosing approaches that are likely to help.

Different situations call for different sorts of strategies. If a problem is confusing, you probably don't have a simple playbook for dealing with it. Different people also benefit from different sorts of strategies. So, while I can tell you a list of potential mental tools, what I most want you to practice is the art of identifying what would help you, in [...]

---

Outline:

(03:54) Example: Writing this Blogpost

(07:36) Suggested Exercises

(09:28) Appendix: Existing available puzzles/games Ive used

(09:35) Thinking Physics

(10:02) Games

---

First published:

Source:

Narrated by TYPE III AUDIO.