Telling the truth is hard. Sometimes you don’t know what's true, sometimes you get confused, and sometimes you really don’t wanna cause lying can get you more cookies reward. It turns out this is true for both humans and AIs!

Now, it matters if an AI (or human) says false things on purpose or by accident. If it's an accident, then we can probably fix that over time. All current AIs make mistakes and all they all make things up - some of the time at least. But do any of them really lie on purpose?

It seems like yes, sometimes they do. There have been experiments that show models express an intent to lie in their chain of thought and then they go ahead and do that. This is rare though. More commonly we catch them saying such clearly self-serving falsehoods that if they were human, we’d still call foul whether they did it “intentionally” or not.

Yet as valuable as it is to detect lies, it remains inherently hard to do so. We’ve run 16 models for dozens to hundreds of hours in the AI Village and haven’t noticed a single “smoking gun”: where an agent expresses [...]

---

Outline:

(02:05) Clauding the Truth

(11:21) o3: Our Factless Leader

(12:17) Rampant Placeholder Expansion

(13:48) Assumption of Ownership

(16:12) What is Truth Even? Over- and Underreporting in the Village

(20:38) So do LLMs lie in the Village?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.