LessWrong (Curated & Popular)

By LessWrongSociety & Culture Technology Philosophy

LessWrong (Curated & Popular) episodes

June 07, 2024“Response to Aschenbrenner’s ‘Situational Awareness’” by Rob Bensinger
(Cross-posted from Twitter.)

My take on Leopold Aschenbrenner's new report: I think Leopold gets it right on a bunch of important counts.

Three that I especially care about:

Full AGI and ASI soon. (I think his arguments for this have a lot of holes, but he gets the basic point that superintelligence looks 5 or 15 years off rather than 50+.)
This technology is an overwhelmingly huge deal, and if we play our cards wrong we're all dead.
Current developers are indeed fundamentally unserious about the core risks, and need to make IP security and closure a top priority.
I especially appreciate that the report seems to get it when it comes to our basic strategic situation: it gets that we may only be a few years away from a truly world-threatening technology, and it speaks very candidly about the implications of this, rather than soft-pedaling [...]

---

First published:
June 6th, 2024

Source:
https://www.lesswrong.com/posts/Yig9oa4zGE97xM2os/response-to-aschenbrenner-s-situational-awareness

---

Narrated by TYPE III AUDIO.
6 min
June 07, 2024 “Humming is not a free $100 bill” by Elizabeth

Last month I posted about humming as a cheap and convenient way to flood your nose with nitric oxide (NO), a known antiviral. Alas, the economists were right, and the benefits were much smaller than I estimated.

The post contained one obvious error and one complication. Both were caught by Thomas Kwa, for which he has my gratitude. When he initially pointed out the error I awarded him a $50 bounty; now that the implications are confirmed I’ve upped that to $250. In two weeks an additional $750 will go to either him or to whoever provides new evidence that causes me to retract my retraction.

Humming produces much less nitric oxide than Enovid

I found the dosage of NO in Enovid in a trial registration. Unfortunately I misread the dose- what I original read as “0.11ppm NO/hour” was in fact “0.11ppm NO*hour”. I [...]

---

First published:
June 6th, 2024

Source:
https://www.lesswrong.com/posts/dsZeogoPQbF8jSHMB/humming-is-not-a-free-usd100-bill

---

Narrated by TYPE III AUDIO.

6 min
June 06, 2024“Announcing ILIAD — Theoretical AI Alignment Conference ” by Nora_Ammann, Alexander Gietelink Oldenziel
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.We are pleased to announce ILIAD — a 5-day conference bringing together 100+ researchers to build strong scientific foundations for AI alignment.

***Apply to attend by June 30!***

When: Aug 28 - Sep 3, 2024
Where: @Lighthaven (Berkeley, US)
What: A mix of topic-specific tracks, and unconference style programming, 100+ attendees. Topics will include Singular Learning Theory, Agent Foundations, Causal Incentives, Computational Mechanics and more to be announced.
Who: Currently confirmed speakers include: Daniel Murfet, Jesse Hoogland, Adam Shai, Lucius Bushnaq, Tom Everitt, Paul Riechers, Scott Garrabrant, John Wentworth, Vanessa Kosoy, Fernando Rosas and James Crutchfield.
Costs: Tickets are free. Financial support is available on a needs basis.
See our website here. For any questions, email [email protected]

About ILIAD

ILIAD is a 100+ person conference about alignment with a mathematical focus. The theme is ecumenical. [...]

---

First published:
June 5th, 2024

Source:
https://www.lesswrong.com/posts/r7nBaKy5Ry3JWhnJT/announcing-iliad-theoretical-ai-alignment-conference

---

Narrated by TYPE III AUDIO.
5 min
May 31, 2024 “Non-Disparagement Canaries for OpenAI” by aysja, Adam Scholl
Since at least 2017, OpenAI has asked departing employees to sign offboarding agreements which legally bind them to permanently—that is, for the rest of their lives—refrain from criticizing OpenAI, or from otherwise taking any actions which might damage its finances or reputation.[1]

If they refused to sign, OpenAI threatened to take back (or make unsellable) all of their already-vested equity—a huge portion of their overall compensation, which often amounted to millions of dollars. Given this immense pressure, it seems likely that most employees signed.

If they did sign, they became personally liable forevermore for any financial or reputational harm they later caused. This liability was unbounded, so had the potential to be financially ruinous—if, say, they later wrote a blog post critical of OpenAI, they might in principle be found liable for damages far in excess of their net worth.

These extreme provisions allowed OpenAI to systematically silence criticism [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
May 30th, 2024

Source:
https://www.lesswrong.com/posts/yRWv5kkDD4YhzwRLq/non-disparagement-canaries-for-openai

---

Narrated by TYPE III AUDIO.

5 min
May 30, 2024 “MIRI 2024 Communications Strategy” by Gretta Duleba
As we explained in our MIRI 2024 Mission and Strategy update, MIRI has pivoted to prioritize policy, communications, and technical governance research over technical alignment research. This follow-up post goes into detail about our communications strategy.

The Objective: Shut it Down[1]

Our objective is to convince major powers to shut down the development of frontier AI systems worldwide before it is too late. We believe that nothing less than this will prevent future misaligned smarter-than-human AI systems from destroying humanity. Persuading governments worldwide to take sufficiently drastic action will not be easy, but we believe this is the most viable path.

Policymakers deal mostly in compromise: they form coalitions by giving a little here to gain a little somewhere else. We are concerned that most legislation intended to keep humanity alive will go through the usual political processes and be ground down into ineffective compromises.

The only way we [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
May 29th, 2024

Source:
https://www.lesswrong.com/posts/tKk37BFkMzchtZThx/miri-2024-communications-strategy

---

Narrated by TYPE III AUDIO.

14 min
May 28, 2024 “OpenAI: Fallout” by Zvi
Previously: OpenAI: Exodus (contains links at top to earlier episodes), Do Not Mess With Scarlett Johansson

We have learned more since last week. It's worse than we knew.

How much worse? In which ways? With what exceptions?

That's what this post is about.

The Story So Far

For years, employees who left OpenAI consistently had their vested equity explicitly threatened with confiscation and the lack of ability to sell it, and were given short timelines to sign documents or else. Those documents contained highly aggressive NDA and non disparagement (and non interference) clauses, including the NDA preventing anyone from revealing these clauses.

No one knew about this until recently, because until Daniel Kokotajlo everyone signed, and then they could not talk about it. Then Daniel refused to sign, Kelsey Piper started reporting, and a lot came out.

Here is Altman's statement from [...]

---

First published:
May 28th, 2024

Source:
https://www.lesswrong.com/posts/YwhgHwjaBDmjgswqZ/openai-fallout

---

Narrated by TYPE III AUDIO.

1 hr 7 min
May 28, 2024 [HUMAN VOICE] Update on human narration for this podcast
Contact: patreon.com/lwcurated or [perrin dot j dot walker plus lesswrong fnord gmail].

All Solenoid's narration work found here.
2 min
May 28, 2024“Maybe Anthropic’s Long-Term Benefit Trust is powerless” by Zach Stein-Perlman
Crossposted from AI Lab Watch. Subscribe on Substack.

Introduction.

Anthropic has an unconventional governance mechanism: an independent "Long-Term Benefit Trust" elects some of its board. Anthropic sometimes emphasizes that the Trust is an experiment, but mostly points to it to argue that Anthropic will be able to promote safety and benefit-sharing over profit.[1]

But the Trust's details have not been published and some information Anthropic has shared is concerning. In particular, Anthropic's stockholders can apparently overrule, modify, or abrogate the Trust, and the details are unclear.

Anthropic has not publicly demonstrated that the Trust would be able to actually do anything that stockholders don't like.

The facts

There are three sources of public information on the Trust:

The Long-Term Benefit Trust (Anthropic 2023)
Anthropic Long-Term Benefit Trust (Morley et al. 2023)
The $1 billion gamble to ensure AI doesn't destroy humanity (Vox: Matthews 2023)
They say there's [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
May 27th, 2024

Source:
https://www.lesswrong.com/posts/sdCcsTt9hRpbX6obP/maybe-anthropic-s-long-term-benefit-trust-is-powerless

---

Narrated by TYPE III AUDIO.
6 min
May 27, 2024 “Notifications Received in 30 Minutes of Class” by tanagrabeast
Introduction.

If you are choosing to read this post, you've probably seen the image below depicting all the notifications students received on their phones during one class period. You probably saw it as a retweet of this tweet, or in one of Zvi's posts. Did you find this data plausible, or did you roll to disbelieve? Did you know that the image dates back to at least 2019? Does that fact make you more or less worried about the truth on the ground as of 2024?

Last month, I performed an enhanced replication of this experiment in my high school classes. This was partly because we had a use for it, partly to model scientific thinking, and partly because I was just really curious. Before you scroll past the image, I want to give you a chance to mentally register your predictions. Did my average class match the [...]

---

First published:
May 26th, 2024

Source:
https://www.lesswrong.com/posts/AZCpu3BrCFWuAENEd/notifications-received-in-30-minutes-of-class

---

Narrated by TYPE III AUDIO.

17 min
May 24, 2024“AI companies aren’t really using external evaluators” by Zach Stein-Perlman
New blog: AI Lab Watch. Subscribe on Substack.

Many AI safety folks think that METR is close to the labs, with ongoing relationships that grant it access to models before they are deployed. This is incorrect. METR (then called ARC Evals) did pre-deployment evaluation for GPT-4 and Claude 2 in the first half of 2023, but it seems to have had no special access since then.[1] Other model evaluators also seem to have little access before deployment.

Frontier AI labs' pre-deployment risk assessment should involve external model evals for dangerous capabilities.[2] External evals can improve a lab's risk assessment and—if the evaluator can publish its results—provide public accountability.

The evaluator should get deeper access than users will get.

To evaluate threats from a particular deployment protocol, the evaluator should get somewhat deeper access than users will — then the evaluator's failure to elicit dangerous capabilities is stronger evidence [...]
The original text contained 5 footnotes which were omitted from this narration.

---

First published:
May 24th, 2024

Source:
https://www.lesswrong.com/posts/WjtnvndbsHxCnFNyc/ai-companies-aren-t-really-using-external-evaluators

---

Narrated by TYPE III AUDIO.
8 min

About LessWrong (Curated & Popular)

From the publisher's feed

Audio narrations of LessWrong posts. Includes all curated posts and all posts with 125+ karma.

If you'd like more, subscribe to the “Lesswrong (30+ karma)” feed.