Sign up to save your podcasts
Or
Share “Fields that I reference when thinking about AI takeover prevention” by Buck Shlegeris
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Fields that I reference when thinking about AI takeover prevention” by Buck Shlegeris
Subtitle: Is AI takeover like a nuclear meltdown? A coup? A plane crash?.
My day job is thinking about safety measures that aim to reduce catastrophic risks from AI (especially risks from egregious misalignment). The two main themes of this work are the design of such measures (what's the space of techniques we might expect to be affordable and effective) and their evaluation (how do we decide which safety measures to implement, and whether a set of measures is sufficiently robust). I focus especially on AI control, where we assume our models are trying to subvert our safety measures and aspire to find measures that are robust anyway.
Like other AI safety researchers, I often draw inspiration from other fields that contain potential analogies. Here are some of those fields, my opinions on their strengths and weaknesses as analogies, and some of my favorite resources on them.
Robustness [...]
---
Outline:
(01:08) Robustness to insider threats
(07:34) Computer security
(10:21) Adversarial risk analysis
(12:25) Safety engineering
(14:07) Physical security
(19:00) How human power structures arise and are preserved
---
First published:
August 13th, 2024
Source:
https://redwoodresearch.substack.com/p/fields-that-i-reference-when-thinking
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By Redwood ResearchSubtitle: Is AI takeover like a nuclear meltdown? A coup? A plane crash?.
My day job is thinking about safety measures that aim to reduce catastrophic risks from AI (especially risks from egregious misalignment). The two main themes of this work are the design of such measures (what's the space of techniques we might expect to be affordable and effective) and their evaluation (how do we decide which safety measures to implement, and whether a set of measures is sufficiently robust). I focus especially on AI control, where we assume our models are trying to subvert our safety measures and aspire to find measures that are robust anyway.
Like other AI safety researchers, I often draw inspiration from other fields that contain potential analogies. Here are some of those fields, my opinions on their strengths and weaknesses as analogies, and some of my favorite resources on them.
Robustness [...]
---
Outline:
(01:08) Robustness to insider threats
(07:34) Computer security
(10:21) Adversarial risk analysis
(12:25) Safety engineering
(14:07) Physical security
(19:00) How human power structures arise and are preserved
---
First published:
August 13th, 2024
Source:
https://redwoodresearch.substack.com/p/fields-that-i-reference-when-thinking
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.