I contributed one (1) task to HCAST, which was used in METR's Long Tasks paper. This gave me some thoughts I feel moved to share.

Regarding Baselines and Estimates

METR's tasks have two sources for how long they take humans: most of those used in the paper were Baselined using playtesters under persistent scrutiny, and some were Estimated by METR.

I don’t quite trust the Baselines. Baseliners were allowed/incentivized to drop tasks they weren’t making progress with, and were – mostly, effectively, there's some nuance here I’m ignoring – cut off at the eight-hour mark; Baseline times were found by averaging time taken for successful runs; this suggests Baseline estimates will be biased to be at least slightly too low, especially for more difficult tasks.[1]

I really, really don’t trust the Estimates[2]. My task was never successfully Baselined, so METR's main source for how long it would take – [...]

---

Outline:

(00:22) Regarding Baselines and Estimates

(02:23) Regarding Task Privacy

(04:00) In Conclusion

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Utilitarianism implies that if we build an AI that successfully maximizes utility/value, we should be ok with it replacing us. Sensible people add caveats related to how hard it’ll be to determine the correct definition of value or check whether the AI is truly optimizing it.

As someone who often passionately rants against the AI successionist line of thinking, the most common objection I hear is "why is your definition of value so arbitrary as to stipulate that biological meat-humans are necessary" This is missing the crux—I agree such a definition of moral value would be hard to justify.

Instead, my opposition to AI successionism comes from a preference toward my own kind. This is hardwired in me from biology. I prefer my family members to randomly-sampled people with similar traits. I would certainly not elect to sterilize or kill my family members so that they could be replaced [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

AI 2027 is a Bet Against Amdahl's Law was my attempt to summarize and analyze "the key load-bearing arguments AI 2027 presents for short timelines". There were a lot of great comments – every time I post on LW is a learning experience. In this post, I'm going to summarize the comments and present some resulting updates to my previous analysis. I'm also using this post to address some comments that I didn't respond to in the original post, because the comment tree was becoming quite sprawling.

TL;DR: my previous post reflected a few misunderstandings of the AI 2027 model, in particular in how to interpret "superhuman AI researcher". Intuitively, I still have trouble accepting the very high speedup factors contemplated in the model, but this could be a failure of my intuition, and I don't have strong evidence to present. New cruxes:

1. The rate of progress from [...]

---

Outline:

(01:27) Confusion Regarding Milestone Definitions

(05:23) Someone Should Flesh Out What Goes Into AI R&D

(09:35) How Long Will it Take To Reach the Early Milestones?

(13:16) Broad Progress on Real-World Tasks Is a Crux

(15:50) Does Hofstadters Law Apply?

(19:46) What Would Be the Impact of an SAR / SIAR?

(22:05) Conclusions

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Strength

In 1997, with Deep Blue's defeat of Kasparov, computers surpassed human beings at chess. Other games have fallen in more recent years: Go, Starcraft, and League of Legends among them. AI is superhuman at these pursuits, and unassisted human beings will never catch up. The situation looks like this:[1]

The average serious chess player is pretty good (1500), the very best chess player is extremely good (2837), and the best AIs are way, way better (3700). Even Deep Blue's estimated Elo is about 2850 - it remains competitive with the best humans alive.

A natural way to describe this situation is to say that AI is superhuman at chess. No matter how you slice it, that's true.

For other activities, though, it's a lot murkier. Take radiology, for example:

---

Outline:

(00:10) Strength

(02:28) Effort

(04:35) And More

(06:36) Beyond Superhuman

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(Disclaimer: Post written in a personal capacity. These are personal hot takes and do not in any way represent my employer's views.)

TL;DR: I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise. Interpretability seems a valuable tool here and remains worth investing in, as it will hopefully increase the reliability we can achieve. However, interpretability should be viewed as part of an overall portfolio of defences: a layer in a defence-in-depth strategy. It is not the one thing that will save us, and it still won’t be enough for high reliability.

Introduction

There's a common, often implicit, argument made in AI safety discussions: interpretability is presented as the only reliable path forward for detecting deception in advanced AI - among many other sources it was argued for in [...]

---

Outline:

(00:55) Introduction

(02:57) High Reliability Seems Unattainable

(05:12) Why Won't Interpretability be Reliable?

(07:47) The Potential of Black-Box Methods

(08:48) The Role of Interpretability

(12:02) Conclusion

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Cryonics Institute and Suspended Animation now have an arrangement where Suspended Animation will conduct a field cryopreservation before shipping the body to Cryonics Institute, thus decreasing tissue damage occuring in transit. They are raising their prices accordingly, but offering a discount from the new price for people who sign up by May 21 (and arrange funding within another 3 months). See https://cryonics.org/members/standby/suspended-animation-inc-standby-transport-services-option/ for details.

It is thus an especially good time to sign up cryonics if you intend to contract with Cryonics Institute plus Suspended Animation, and live in the United States. If you don't like in the US, don't intend to contract with CI, or intend to contract with CI but not also with SA, then this deadline doesn't mean anything for you, but, if you want to, you could still take this as impetus to get around to signing up.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Politico writes:

The [Ukrainian] program […] rewards soldiers with points if they upload videos proving their drones have hit Russian targets. It will soon be integrated with a new online marketplace called Brave 1 Market, which will allow troops to convert those points into new equipment for their units.

[...]

The program assigns points for each type of kill: 20 points for damaging and 40 for destroying a tank; up to 50 points for destroying a mobile rocket system, depending on the caliber; and six points for killing an enemy soldier.

[...]

Units will soon be able to use the special digital points they’ve been getting since last year by trading them in for new weapons. A Vampire drone, for example, costs 43 points. The drone, nicknamed Baba Yaga, or witch, is a large multi-rotor drone able to carry a 15-kilogram warhead. The Ukrainian government will pay for the [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Burnout. Burn out? Whatever, it sucks.

Burnout is a pretty confusing thing made harder by our naive reactions being things like “just try harder” or “grit your teeth and push through”, which usually happen to be exactly the wrong things to do. Burnout also isn’t really just one thing, it's more like a collection of distinct problems that are clustered by similar symptoms.

Something something intro, research context, this is what I’ve learned / synthesized blah blah blah. Read on!

Models of burnout

These are models of burnout that I’ve found particularly useful, with the reminder that these are just models with all the caveats that that comes with.

Burnout as a mental injury

Researchers can be thought of as “mental athletes” who get “mental injuries” (such as burnout) the way physical athletes get physical injuries, and we should orient towards these mental injuries in the same way [...]

---

Outline:

(00:41) Models of burnout

(00:52) Burnout as a mental injury

(02:17) Burnout as a deficit of activation energy

(03:25) Sources of burnout

(04:23) Physiological + Psychological

(04:56) Broken steering / responsiveness

(06:17) Permanent on-call

(07:04) Mission doubt

(08:19) Lightness and heaviness

(10:06) Early warning signs

(11:42) Coping mechanisms and solutions

---

First published:

Source:

Narrated by TYPE III AUDIO.

As an employee of the European AI Office, it's important for me to emphasize this point: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions.

Also, to stave off a common confusion: I worked at ARC Theory, which is now simply called ARC, on Paul Christiano's theoretical alignment agenda. The more famous ARC Evals was a different group working on evaluations, their work was completely separate from ARC Theory, and they were only housed under the same organization out of convenience, until ARC Evals spun off under the name METR. Nothing I write here has any implication about the work of ARC Evals/METR in any way.

Low Probability Estimation

This is my third post in a sequence of posts on ARC's agenda, you should definitely read the first post before this one for [...]

---

Outline:

(00:56) Low Probability Estimation

(02:42) LPE on real distributions

(04:41) LPE as training signal

(07:55) Does LPE work at all?

The original text contained 11 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

As an employee of the European AI Office, it's important for me to emphasize this point: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions.

Also, to stave off a common confusion: I worked at ARC Theory, which is now simply called ARC, on Paul Christiano's theoretical alignment agenda. The more famous ARC Evals was a different group working on evaluations, their work was completely separate from ARC Theory, and they were only housed under the same organization out of convenience, until ARC Evals spun off under the name METR. Nothing I write here has any implication about the work of ARC Evals/METR in any way.

Mechanistic Anomaly Detection

This is my second post in a sequence of posts on ARC's agenda. You should read the first post before this one for context.

[...]

---

Outline:

(00:55) Mechanistic Anomaly Detection

(02:54) Special case: Safe Distillation

(07:29) General case: Handling out of distribution events

(10:05) MAD solution proposal: The fragility of sensor tampering

(16:02) Detecting high-stakes failures

The original text contained 10 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.