LessWrong (Curated & Popular)

A key question going forward is the extent to which making further AI progress will depend upon some form of continual learning. Dwarkesh Patel offers us an extended essay considering these questions and reasons to be skeptical of the pace of progress for a while. I am less skeptical about many of these particular considerations, and do my best to explain why in detail.

Separately, Ivanka Trump recently endorsed a paper with a discussion I liked a lot less but that needs to be discussed given how influential her voice might (mind you I said might) be to policy going forward, so I will then cover that here as well.

Dwarkesh Patel on Continual Learning

Dwarkesh Patel explains why he doesn’t think AGI is right around the corner, and why AI progress today is insufficient to replace most white collar employment: That continual learning is both [...]

---

Outline:

(00:44) Dwarkesh Patel on Continual Learning

(09:43) Comparing Investments

(11:51) Comparing Predictions

(21:18) Others React to Dwarkesh Patel

(28:25) Ivanka Trump and The Era of Experience

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/YEwzhjFzt3zKctg2F/dwarkesh-patel-on-continual-learning

---

Narrated by TYPE III AUDIO.

Just posted the following on Medium. Interested in comments from readers here, especially pointers to similar efforts and ideas I didn't mention below.

This is the first in a series of articles describing features, functions, and components of Personal Agents — next generation AI virtual assistants that will serve as trusted advisors, caretakers, and user proxies. Personal Agents will preferably be developed as an open source project. Primary goals are to specify agents that (1) Significantly benefit people (are not just cool or fun) and (2) Avoid harmful side-effects (like those plaguing social media or that worry AI safety advocates). A clear and open specification will facilitate agent development and certification. This article provides a brief overview of Personal Agents.

Personal Agents (PAs), introduced here and here, are next-generation virtual assistants[1] that will support people in all aspects of their lives — from health and safety to education, career [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/tJg9AxhgsZpizeGv5/personal-agents-ais-as-trusted-advisors-caretakers-and-user

---

Narrated by TYPE III AUDIO.

This is a link post. METR just made a lovely post detailing many examples they've found of reward hacks by frontier models. Unlike the reward hacks of yesteryear, these models are smart enough to know that what they are doing is deceptive and not what the company wanted them to do.

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/Zu4ai9GFpwezyfB2K/metr-recent-frontier-models-are-reward-hacking

Linkpost URL:
https://metr.org/blog/2025-06-05-recent-reward-hacking/

---

Narrated by TYPE III AUDIO.

This is a link post. Using representation engineering, we systematically induce, detect, and control such deception in CoT-enabled LLMs, extracting ”deception vectors” via Linear Artificial Tomography (LAT) for 89% detection accuracy. Through activation steering, we achieve a 40% success rate in eliciting context-appropriate deception without explicit prompts, unveiling the specific honesty related issue of reasoning models and providing tools for trustworthy AI alignment.



This seems like a positive breakthrough for mech interp research generally, the team used RepE to identify features, and were able to "reliably suppress or induce strategic deception".

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/3WyFmtiLZTfEQxJCy/identifying-deception-vectors-in-models

Linkpost URL:
https://arxiv.org/pdf/2506.04909

---

Narrated by TYPE III AUDIO.

Crosspost from my blog.

I just got back from Effective Altruism Global London—a conference that brought together lots of different people trying to do good with their money and careers. It was an inspiring experience. When you write about factory farming, insect suffering, global poverty, and the torment of shrimp, it can, as I’ve mentioned before, feel like screaming into the void. When you try to explain why it's important that we don’t torture insects by the trillions in insect farms, most people look at you like you’ve grown a third head (after the second head that they look at you like you’ve grown when you started talking about shrimp welfare).

But at effective altruism conferences, people actually care. They’re not indifferent to most of the world's suffering. They don’t think I’m crazy! There are other people who think the suffering of animals matters—even the suffering of [...]



---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/zTF5idEcK5frNivBt/the-unparalleled-awesomeness-of-effective-altruism

---

Narrated by TYPE III AUDIO.

---

Images from the article:

As I ease out into a short sabbatical, I find myself turning back to dig the seeds of my repeated cycle of exhaustion and burnout in the last few years.

Many factors were at play, some more personal that I’m comfortable discussing here. But I have unearthed at least one failure mode that I see reflected and diffracted in others lives, especially people who like me love to think, to make sense, to understand. So that seems worth a blog post, if only to plant a pointer to the problem, and my own way to solve it.

I’ve christened this issue the “true goal fallacy”: the unchecked yet embodied assumption that there is a correct goal in the world, a true essence in need of discovery and revealing.

Case Study: Team Lead Crash

A concrete example: the inciting incident of my first burnout was my promotion to team lead.

[...]

---

Outline:

(00:48) Case Study: Team Lead Crash

(03:48) Axiology of True Goal Fallacy

(06:20) Absorbing Ambiguity And Allowing Feedback

(11:55) On Never Being Fully Cured

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/B4zKRZh5oxyGnAdos/the-true-goal-fallacy

---

Narrated by TYPE III AUDIO.

AI companies claim that their models are safe on the basis of dangerous capability evaluations. OpenAI, Google DeepMind, and Anthropic publish reports intended to show their eval results and explain why those results imply that the models' capabilities aren't too dangerous.[1] Unfortunately, the reports mostly don't support the companies' claims. Crucially, the companies usually don't explain why they think the results, which often seem strong, actually indicate safety, especially for biothreat and cyber capabilities. (Additionally, the companies are undereliciting and thus underestimating their models' capabilities, and they don't share enough information for people on the outside to tell how bad this is.)

Bad explanation/contextualization

OpenAI biothreat evals: OpenAI says "several of our biology evaluations indicate our models are on the cusp of being able to meaningfully help novices create known biological threats, which would cross our high risk threshold." It doesn't say how it concludes this (or what results [...]

---

Outline:

(00:54) Bad explanation/contextualization

(04:34) Dubious elicitation

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/AK6AihHGjirdoiJg6/ai-companies-eval-reports-mostly-don-t-support-their-claims

---

Narrated by TYPE III AUDIO.

People sometimes wonder whether certain AIs or animals are conscious/sentient/sapient/have qualia/etc. I don't think that such questions are coherent. Consciousness is a concept that humans developed for reasoning about humans. It's a useful concept, not because it is ontologically fundamental, but because different humans have lots of close similarities in how our cognition works, and we have privileged access to some details of our own cognition, so “it's like what's going on in my head, but with some differences that I can infer from the fact that they don't act quite the same way I do” is a useful way to understand what's going on in other peoples' heads, and we use consciousness-related language to describe features of human minds that we can understand this way. Consciousness is the thing that a typical adult human recognizes in themselves when hearing others describe the character of their internal cognition. This [...]

---

First published:
June 9th, 2025

Source:
https://www.lesswrong.com/posts/q9A9ZFqW3dDbcTBQL/against-asking-if-ais-are-conscious

---

Narrated by TYPE III AUDIO.

Four agents woke up with four computers, a view of the world wide web, and a shared chat room full of humans. Like Claude plays Pokemon, you can watch these agents figure out a new and fantastic world for the first time. Except in this case, the world they are figuring out is our world.

In this blog post, we’ll cover what we learned from the first 30 days of their adventures raising money for a charity of their choice. We’ll briefly review how the Agent Village came to be, then what the various agents achieved, before discussing some general patterns we have discovered in their behavior, and looking toward the future of the project.

Building the Village The Agent Village is an idea by Daniel Kokotajlo where he proposed giving 100 agents their own computer, and letting each pursue their own goal, in their own way, according to [...]

---

Outline:

(00:50) Building the Village

(02:26) Meet the Agents

(08:52) Collective Agent Behavior

(12:26) Future of the Village

---

First published:
May 27th, 2025

Source:
https://www.lesswrong.com/posts/jyrcdykz6qPTpw7FX/season-recap-of-the-village-agents-raise-usd2-000

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Introduction

The Best Textbooks on Every Subject is the Schelling point for the best textbooks on every subject. My The Best Tacit Knowledge Videos on Every Subject is the Schelling point for the best tacit knowledge videos on every subject. This post is the Schelling point for the best reference works for every subject.

Reference works provide an overview of a subject. Types of reference works include charts, maps, encyclopedias, glossaries, wikis, classification systems, taxonomies, syllabi, and bibliographies.

Reference works are valuable for orienting oneself to fields, particularly when beginning. They can help identify unknown unknowns; they help get a sense of the bigger picture; they are also very interesting and fun to explore.

How to Submit

My previous The Best Tacit Knowledge Videos on Every Subject uses author credentials to assess the epistemics of submissions. The Best Textbooks on Every Subject requires submissions to be from someone who [...]

---

Outline:

(00:10) Introduction

(01:00) How to Submit

(02:15) The List

(02:18) Humanities

(02:21) History

(03:46) Religion

(04:02) Philosophy

(04:29) Literature

(04:43) Formal Sciences

(04:47) Computer Science

(05:16) Mathematics

(05:59) Natural Sciences

(06:02) Physics

(06:16) Earth Science

(06:33) Astronomy

(06:47) Professional and Applied Sciences

(06:51) Library and Information Sciences

(07:34) Education

(08:00) Research

(08:32) Finance

(08:51) Medicine and Health

(09:21) Meditation

(09:52) Urban Planning

(10:24) Social Sciences

(10:27) Economics

(10:39) Political Science

(10:54) By Medium

(11:21) Other Lists like This

(12:41) Further Reading

---

First published:
May 14th, 2025

Source:
https://www.lesswrong.com/posts/HLJMyd4ncE3kvjwhe/the-best-reference-works-for-every-subject

---

Narrated by TYPE III AUDIO.