Sign up to save your podcasts
Or
Share [Linkpost] “Modifying LLM Beliefs with Synthetic Document Finetuning” by RowanWang, Johannes Treutlein, Ethan Perez, Fabien Roger, Sam Marks
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
[Linkpost] “Modifying LLM Beliefs with Synthetic Document Finetuning” by RowanWang, Johannes Treutlein, Ethan Perez, Fabien Roger, Sam Marks
This is a link post.
In this post, we study whether we can modify an LLM's beliefs and investigate whether doing so could decrease risk from advanced AI systems.
We describe a pipeline for modifying LLM beliefs via synthetic document finetuning and introduce a suite of evaluations that suggest our pipeline succeeds in inserting all but the most implausible beliefs. We also demonstrate proof-of-concept applications to honeypotting for detecting model misalignment and unlearning.
Introduction:
Large language models develop implicit beliefs about the world during training, shaping how they reason and actIn this work, we construe AI systems as believing in a claim if they consistently behave in accordance with that claim. In this work, we study whether we can systematically modify these beliefs, creating a powerful new affordance for safer AI deployment.
Controlling the beliefs of AI systems can decrease risk in a variety of ways. First, model organisms [...]
---
First published:
April 24th, 2025
Source:
https://www.lesswrong.com/posts/ARQs7KYY9vJHeYsGc/untitled-draft-2qxt
Linkpost URL:
https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongThis is a link post.
In this post, we study whether we can modify an LLM's beliefs and investigate whether doing so could decrease risk from advanced AI systems.
We describe a pipeline for modifying LLM beliefs via synthetic document finetuning and introduce a suite of evaluations that suggest our pipeline succeeds in inserting all but the most implausible beliefs. We also demonstrate proof-of-concept applications to honeypotting for detecting model misalignment and unlearning.
Introduction:
Large language models develop implicit beliefs about the world during training, shaping how they reason and actIn this work, we construe AI systems as believing in a claim if they consistently behave in accordance with that claim. In this work, we study whether we can systematically modify these beliefs, creating a powerful new affordance for safer AI deployment.
Controlling the beliefs of AI systems can decrease risk in a variety of ways. First, model organisms [...]
---
First published:
April 24th, 2025
Source:
https://www.lesswrong.com/posts/ARQs7KYY9vJHeYsGc/untitled-draft-2qxt
Linkpost URL:
https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,367 Listeners
Conversations with Tyler
2,397 Listeners
The Peter Attia Drive
7,779 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,103 Listeners
ManifoldOne
87 Listeners
Your Undivided Attention
1,442 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
8,778 Listeners
Machine Learning Street Talk (MLST)
89 Listeners
Dwarkesh Podcast
355 Listeners
Hard Fork
5,370 Listeners
The Ezra Klein Show
15,053 Listeners
Moonshots with Peter Diamandis
460 Listeners
No Priors: Artificial Intelligence | Technology | Startups
126 Listeners
Latent Space: The AI Engineer Podcast
64 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
432 Listeners