LessWrong (30+ Karma)

“Clarifying METR’s Auditing Role” by Beth Barnes


Listen Later

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

Although METR has never claimed to have audited anything or to be providing meaningful oversight or accountability, there has been some confusion about whether METR is an auditor or planning to be one.

To clarify this point:

  1. METR's top priority is to develop the science of evaluations, and we don’t need to be auditors in order to succeed at this.
    • We aim to build evaluation protocols that can be used by evaluators/auditors regardless of whether that is the government, an internal lab team, another third party, or a team at METR. 
       
  2. We should not be considered to have ‘audited’ GPT-4 or Claude.
    • Those were informal pilots of what an audit might involve, or research collaborations – not providing meaningful oversight. For example, it was all under NDA – we didn’t have [...]



---

First published:

May 30th, 2024

Source:

https://www.lesswrong.com/posts/yHFhWmu3DmvXZ5Fsm/clarifying-metr-s-auditing-role

---

Narrated by TYPE III AUDIO.

...more
View all episodesView all episodes
Download on the App Store

LessWrong (30+ Karma)By LessWrong


More shows like LessWrong (30+ Karma)

View all
The Daily by The New York Times

The Daily

112,909 Listeners

Astral Codex Ten Podcast by Jeremiah

Astral Codex Ten Podcast

130 Listeners

Interesting Times with Ross Douthat by New York Times Opinion

Interesting Times with Ross Douthat

7,215 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

532 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

16,221 Listeners

AI Article Readings by Readings of great articles in AI voices

AI Article Readings

4 Listeners

Doom Debates by Liron Shapira

Doom Debates

14 Listeners

LessWrong posts by zvi by zvi

LessWrong posts by zvi

2 Listeners