LessWrong (Curated & Popular)

By LessWrongSociety & Culture Technology Philosophy

LessWrong (Curated & Popular) episodes

January 05, 2024 MIRI 2024 Mission and Strategy Update
As we announced back in October, I have taken on the senior leadership role at MIRI as its CEO. It's a big pair of shoes to fill, and an awesome responsibility that I’m honored to take on.

There have been several changes at MIRI since our 2020 strategic update, so let's get into it.[1]

The short version:

We think it's very unlikely that the AI alignment field will be able to make progress quickly enough to prevent human extinction and the loss of the future's potential value, that we expect will result from loss of control to smarter-than-human AI systems.

However, developments this past year like the release of ChatGPT seem to have shifted the Overton window in a lot of groups. There's been a lot more discussion of extinction risk from AI, including among policymakers, and the discussion quality seems greatly improved.

This provides a glimmer of hope. [...]

The original text contained 7 footnotes which were omitted from this narration.

---

First published:
January 5th, 2024

Source:
https://www.lesswrong.com/posts/q3bJYTB3dGRf5fbD9/miri-2024-mission-and-strategy-update

---

Narrated by TYPE III AUDIO.

15 min
January 04, 2024The Plan - 2023 Version
Background: The Plan, The Plan: 2022 Update. If you haven’t read those, don’t worry, we’re going to go through things from the top this year, and with moderately more detail than before.

1. What's Your Plan For AI Alignment?

Median happy trajectory:

Sort out our fundamental confusions about agency and abstraction enough to do interpretability that works and generalizes robustly.
Look through our AI's internal concepts for a good alignment target, then Retarget the Search [1].
…
Profit!
We’ll talk about some other (very different) trajectories shortly.

A side-note on how I think about plans: I’m not really optimizing to make the plan happen. Rather, I think about many different “plans” as possible trajectories, and my optimization efforts are aimed at robust bottlenecks - subproblems which are bottlenecks on lots of different trajectories. An example from the linked post:

For instance, if I wanted to build a solid-state amplifier in [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
December 29th, 2023

Source:
https://www.lesswrong.com/posts/HfqbjwpAEGep9mHhc/the-plan-2023-version

---

Narrated by TYPE III AUDIO.
58 min
January 03, 2024Apologizing is a Core Rationalist Skill
In certain circumstances, apologizing can also be a countersignalling power-move, i.e. “I am so high status that I can grovel a bit without anybody mistaking me for a general groveller”. But that's not really the type of move this post is focused on.There's this narrative about a tradeoff between:

The virtue of Saying Oops, early and often, correcting course rather than continuing to pour oneself into a losing bet, vs
The loss of social status one suffers by admitting defeat, rather than spinning things as a win or at least a minor setback, or defending oneself.
In an ideal world - goes the narrative - social status mechanisms would reward people for publicly updating, rather than defending or spinning their every mistake. But alas, that's not how the world actually works, so as individuals we’re stuck making difficult tradeoffs.

I claim that this narrative is missing a key piece. [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
January 2nd, 2024

Source:
https://www.lesswrong.com/posts/xiTLBuhEMmoyeor6D/apologizing-is-a-core-rationalist-skill

---

Narrated by TYPE III AUDIO.
11 min
January 02, 2024 [HUMAN VOICE] "A case for AI alignment being difficult" by jessicata
This is a linkpost for https://unstableontology.com/2023/12/31/a-case-for-ai-alignment-being-difficult/

Support ongoing human narrations of LessWrong's curated posts:
www.patreon.com/LWCurated
This is an attempt to distill a model of AGI alignment that I have gained primarily from thinkers such as Eliezer Yudkowsky (and to a lesser extent Paul Christiano), but explained in my own terms rather than attempting to hew close to these thinkers. I think I would be pretty good at passing an ideological Turing test for Eliezer Yudowsky on AGI alignment difficulty (but not AGI timelines), though what I'm doing in this post is not that, it's more like finding a branch in the possibility space as I see it that is close enough to Yudowsky's model that it's possible to talk in the same language.
Even if the problem turns out to not be very difficult, it's helpful to have a model of why one might think it is difficult, so as to identify weaknesses in the case so as to find AI designs that avoid the main difficulties. Progress on problems can be made by a combination of finding possible paths and finding impossibility results or difficulty arguments.
Most of what I say should not be taken as a statement on AGI timelines. Some problems that make alignment difficult, such as ontology identification, also make creating capable AGI difficult to some extent.
Source:
https://www.lesswrong.com/posts/wnkGXcAq4DCgY8HqA/a-case-for-ai-alignment-being-difficult

Narrated for LessWrong by Perrin Walker.

Share feedback on this narration.

[Curated Post] ✓
29 min
January 01, 2024 The Dark Arts
lsusrIt is my understanding that you won all of your public forum debates this year. That's very impressive. I thought it would be interesting to discuss some of the techniques you used.

LyrongolemOf course! So, just for a brief overview for those who don't know, public forum is a 2v2 debate format, usually on a policy topic. One of the more interesting ones has been the last one I went to, where the topic was "Resolved: The US Federal Government Should Substantially Increase its Military Presence in the Arctic".

Now, the techniques I'll go over here are related to this topic specifically, but they would also apply to other forms of debate, and argumentation in general really. For the sake of simplicity, I'll call it "ultra-BS".

So, most of us are familiar with 'regular' BS. The idea is the other person says something, and you just reply [...]

---

First published:
December 19th, 2023

Source:
https://www.lesswrong.com/posts/djWftXndJ7iMPsjrp/the-dark-arts

---

Narrated by TYPE III AUDIO.

19 min
December 28, 2023 Critical review of Christiano’s disagreements with Yudkowsky
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a review of Paul Christiano's article "where I agree and disagree with Eliezer". Written for the LessWrong 2022 Review.

In the existential AI safety community, there is an ongoing debate between positions situated differently on some axis which doesn't have a common agreed-upon name, but where Christiano and Yudkowsky can be regarded as representatives of the two directions[1]. For the sake of this review, I will dub the camps gravitating to the different ends of this axis "Prosers" (after prosaic alignment) and "Poets"[2]. Christiano is a Proser, and so are most people in AI safety groups in the industry. Yudkowsky is a typical Poet, people in MIRI and the agent foundations community tend to also be such.

Prosers tend to be more optimistic, lend more credence to slow takeoff, and place more value on [...]

The original text contained 8 footnotes which were omitted from this narration.

---

First published:
December 27th, 2023

Source:
https://www.lesswrong.com/posts/8HYJwQepynHsRKr6j/critical-review-of-christiano-s-disagreements-with-yudkowsky

---

Narrated by TYPE III AUDIO.

29 min
December 27, 2023 Most People Don’t Realize We Have No Idea How Our AIs Work
This point feels fairly obvious, yet seems worth stating explicitly.

Those of us familiar with the field of AI after the deep-learning revolution know perfectly well that we have no idea how our ML models work. Sure, we have an understanding of the dynamics of training loops and SGD's properties, and we know how ML models' architectures work. But we don't know what specific algorithms ML models' forward passes implement. We have some guesses, and some insights painstakingly mined by interpretability advances, but nothing even remotely like a full understanding.

And most certainly, we wouldn't automatically know how a fresh model trained on a novel architecture that was just spat out by the training loop works.

We're all used to this state of affairs. It's implicitly-assumed shared background knowledge. But it's actually pretty unusual, when you first learn of it.

And...

Relevant XKCD.I'm pretty sure that the general public [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 21st, 2023

Source:
https://www.lesswrong.com/posts/CpjTJtW2RNKvzAehG/most-people-don-t-realize-we-have-no-idea-how-our-ais-work

---

Narrated by TYPE III AUDIO.

3 min
December 26, 2023Discussion: Challenges with Unsupervised LLM Knowledge Discovery
TL;DR: Contrast-consistent search (CCS) seemed exciting to us and we were keen to apply it. At this point, we think it is unlikely to be directly helpful for implementations of alignment strategies (>95%). Instead of finding knowledge, it seems to find the most prominent feature. We are less sure about the wider category of unsupervised consistency-based methods, but tend to think they won’t be directly helpful either (70%). We’ve written a paper about some of our detailed experiences with it.

Paper authors: Sebastian Farquhar*, Vikrant Varma*, Zac Kenton*, Johannes Gasteiger, Vlad Mikulik, and Rohin Shah. *Equal contribution, order randomised.

Credences are based on a poll of Seb, Vikrant, Zac, Johannes, Rohin and show single values where we mostly agree and ranges where we disagreed.

What does CCS try to do?

To us, CCS represents a family of possible algorithms aiming at solving an ELK-style problem that have the steps:

[...]
The original text contained 5 footnotes which were omitted from this narration.

---

First published:
December 18th, 2023

Source:
https://www.lesswrong.com/posts/wtfvbsYjNHYYBmT3k/discussion-challenges-with-unsupervised-llm-knowledge-1

---

Narrated by TYPE III AUDIO.
19 min
December 24, 2023 Succession
This is a linkpost for https://www.narrativeark.xyz/p/succession“A table beside the evening sea
where you sit shelling pistachios,
flicking the next open with the half-
shell of the last, story opening story,
on down to the sandy end of time.”

V1: Leaving

Deceleration is the hardest part. Even after burning almost all of my fuel, I’m still coming in at 0.8c. I’ve planned a slingshot around the galaxy's central black hole which will slow me down even further, but at this speed it’ll require incredibly precise timing. I’ve been optimized hard for this, with specialized circuits for it built in on the hardware level to reduce latency. Even so, less than half of slingshots at this speed succeed—most probes crash, or fly off trajectory and are left coasting through empty space.

I’ve already beaten the odds by making it here. Intergalactic probes travel so fast, and so far, that almost all [...]

---

First published:
December 20th, 2023

Source:
https://www.lesswrong.com/posts/CAzntXYTEaNfC9nB6/succession

Linkpost URL:
https://www.narrativeark.xyz/p/succession

---

Narrated by TYPE III AUDIO.

20 min
December 21, 2023 Nonlinear’s Evidence: Debunking False and Misleading Claims
Recently, Ben Pace wrote a well-intentioned blog post mostly based on complaints from 2 (of 21) Nonlinear employees who 1) wanted more money, 2) felt socially isolated, and 3) felt persecuted/oppressed.

Of relevance, one has accused the majority of her previous employers, and 28 people of abuse - that we know of.

She has accused multiple people of threatening to kill her and literally accused an ex-employer of murder. Within three weeks of joining us, she had accused five separate people of abuse: not paying her what was promised, controlling her romantic life, hiring stalkers, and other forms of persecution.

We have empathy for her. Initially, we believed her too. We spent weeks helping her get her “nefarious employer to finally pay her” and commiserated with her over how badly they mistreated her.

Then she started accusing us of strange things.

You’ve seen Ben's evidence, which [...]

---

First published:
December 12th, 2023

Source:
https://www.lesswrong.com/posts/q4MXBzzrE6bnDHJbM/nonlinear-s-evidence-debunking-false-and-misleading-claims

---

Narrated by TYPE III AUDIO.

1 hr

About LessWrong (Curated & Popular)

From the publisher's feed

Audio narrations of LessWrong posts. Includes all curated posts and all posts with 125+ karma.

If you'd like more, subscribe to the “Lesswrong (30+ karma)” feed.