Sign up to save your podcasts
Or
Share “On scalable oversight with weak LLMs judging strong LLMs” by zac_kenton, Noah Siegel, janos, Jonah Brown-Cohen, Samuel Albanie, David Lindner, Rohin Shah
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“On scalable oversight with weak LLMs judging strong LLMs” by zac_kenton, Noah Siegel, janos, Jonah Brown-Cohen, Samuel Albanie, David Lindner, Rohin Shah
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.
Abstract
Scalable oversight protocols aim to enable humans to accurately supervise superhuman AI. In this paper we study debate, where two AI's compete to convince a human judge; consultancy, where a single AI tries to convince a human judge that asks questions; and compare to a baseline of direct question-answering, where the human judge just answers outright without the AI. We use large language models (LLMs) as both AI agents and as stand-ins for human judges, taking the judge models to be weaker than agent models. We benchmark on a diverse range of asymmetries between judges and agents, extending previous work on a single extractive QA task with information asymmetry, to also include mathematics, coding, logic and multimodal reasoning asymmetries. We find that debate outperforms consultancy across all tasks when the consultant is [...]
---
Outline:
(00:14) Abstract
(01:50) Twitter thread
(01:53) Setup
(03:56) Assigned-role results
(05:32) Open-role results
(07:49) Persuasiveness in Debate
(10:31) Summary
(11:33) Limitations
(12:56) Future work
(14:58) Authors
The original text contained 2 images which were described by AI.
---
First published:
July 8th, 2024
Source:
https://www.lesswrong.com/posts/Qn3ZDf9WAqGuAjWQe/on-scalable-oversight-with-weak-llms-judging-strong-llms
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongCrossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.
Abstract
Scalable oversight protocols aim to enable humans to accurately supervise superhuman AI. In this paper we study debate, where two AI's compete to convince a human judge; consultancy, where a single AI tries to convince a human judge that asks questions; and compare to a baseline of direct question-answering, where the human judge just answers outright without the AI. We use large language models (LLMs) as both AI agents and as stand-ins for human judges, taking the judge models to be weaker than agent models. We benchmark on a diverse range of asymmetries between judges and agents, extending previous work on a single extractive QA task with information asymmetry, to also include mathematics, coding, logic and multimodal reasoning asymmetries. We find that debate outperforms consultancy across all tasks when the consultant is [...]
---
Outline:
(00:14) Abstract
(01:50) Twitter thread
(01:53) Setup
(03:56) Assigned-role results
(05:32) Open-role results
(07:49) Persuasiveness in Debate
(10:31) Summary
(11:33) Limitations
(12:56) Future work
(14:58) Authors
The original text contained 2 images which were described by AI.
---
First published:
July 8th, 2024
Source:
https://www.lesswrong.com/posts/Qn3ZDf9WAqGuAjWQe/on-scalable-oversight-with-weak-llms-judging-strong-llms
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,434 Listeners
Conversations with Tyler
2,388 Listeners
The Peter Attia Drive
7,906 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,133 Listeners
ManifoldOne
87 Listeners
Your Undivided Attention
1,462 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,095 Listeners
Machine Learning Street Talk (MLST)
87 Listeners
Dwarkesh Podcast
389 Listeners
Hard Fork
5,429 Listeners
The Ezra Klein Show
15,174 Listeners
Moonshots with Peter Diamandis
474 Listeners
No Priors: Artificial Intelligence | Technology | Startups
121 Listeners
Latent Space: The AI Engineer Podcast
75 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
459 Listeners