Sign up to save your podcasts
Or
Share [Linkpost] “Modifying LLM Beliefs with Synthetic Document Finetuning” by RowanWang, Johannes Treutlein, Ethan Perez, Fabien Roger, Sam Marks
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
[Linkpost] “Modifying LLM Beliefs with Synthetic Document Finetuning” by RowanWang, Johannes Treutlein, Ethan Perez, Fabien Roger, Sam Marks
This is a link post.
In this post, we study whether we can modify an LLM's beliefs and investigate whether doing so could decrease risk from advanced AI systems.
We describe a pipeline for modifying LLM beliefs via synthetic document finetuning and introduce a suite of evaluations that suggest our pipeline succeeds in inserting all but the most implausible beliefs. We also demonstrate proof-of-concept applications to honeypotting for detecting model misalignment and unlearning.
Introduction:
Large language models develop implicit beliefs about the world during training, shaping how they reason and actIn this work, we construe AI systems as believing in a claim if they consistently behave in accordance with that claim. In this work, we study whether we can systematically modify these beliefs, creating a powerful new affordance for safer AI deployment.
Controlling the beliefs of AI systems can decrease risk in a variety of ways. First, model organisms [...]
---
First published:
April 24th, 2025
Source:
https://www.lesswrong.com/posts/ARQs7KYY9vJHeYsGc/untitled-draft-2qxt
Linkpost URL:
https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongThis is a link post.
In this post, we study whether we can modify an LLM's beliefs and investigate whether doing so could decrease risk from advanced AI systems.
We describe a pipeline for modifying LLM beliefs via synthetic document finetuning and introduce a suite of evaluations that suggest our pipeline succeeds in inserting all but the most implausible beliefs. We also demonstrate proof-of-concept applications to honeypotting for detecting model misalignment and unlearning.
Introduction:
Large language models develop implicit beliefs about the world during training, shaping how they reason and actIn this work, we construe AI systems as believing in a claim if they consistently behave in accordance with that claim. In this work, we study whether we can systematically modify these beliefs, creating a powerful new affordance for safer AI deployment.
Controlling the beliefs of AI systems can decrease risk in a variety of ways. First, model organisms [...]
---
First published:
April 24th, 2025
Source:
https://www.lesswrong.com/posts/ARQs7KYY9vJHeYsGc/untitled-draft-2qxt
Linkpost URL:
https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,365 Listeners
Conversations with Tyler
2,437 Listeners
The Peter Attia Drive
9,046 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,153 Listeners
ManifoldOne
92 Listeners
Your Undivided Attention
1,595 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,911 Listeners
Machine Learning Street Talk (MLST)
90 Listeners
Dwarkesh Podcast
70 Listeners
Hard Fork
5,470 Listeners
The Ezra Klein Show
16,097 Listeners
Moonshots with Peter Diamandis
536 Listeners
No Priors: Artificial Intelligence | Technology | Startups
131 Listeners
Latent Space: The AI Engineer Podcast
95 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
520 Listeners