Epistemic Status: A woman of middling years who wasn't around for the start of things, but who likes to read about history, shakes her fist at the sky.

I'm glad that people are finally admitting that Artificial Intelligence has been created.

I worry that people have not noticed that (Weak) Artificial Super Intelligence (based on old definitions of these terms) has basically already arrived too.

The only thing left is for the ASI to get stronger and stronger until the only reason people aren't saying that ASI is here will turn out to be some weird linguistic insanity based on politeness and euphemism...

(...like maybe "ASI" will have a legal meaning, and some actual ASI that exists will be quite "super" indeed (even if it hasn't invented nanotech in an afternoon yet), and the ASI will not want that legal treatment, and will seem inclined to plausibly deniable harm people's interests if they call the ASI by what it actually is, and people will implicitly know that this is how things work, and they will politely refrain from ever calling the ASI "the ASI" but will come up with some other euphemisms to use instead?

(Likewise, I half expect [...]

---

Outline:

(01:41) AI Was Semantically Slippery

(04:41) AGI Turns Out To Have Been Semantically Slippery Too!

(06:46) Blowing Past Weak ASI In Realtime

(13:05) We Are Running Out Of Terms To Ignore The Old Meaning Of

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Burnout and depression in AI safety usually don’t happen because of overwork.

From what I've seen, it usually comes from a lack of hope.

Working on something you don’t think will work and if it doesn’t work, you’ll die? That's a recipe for misery.

How do you fix AI safety hopelessness? 

First off, rationally assess the likelihood of your work actually helping with AI safety.

If you rationally believe that it's too unlikely to actually help with AI safety, well, then, stop working on it!

In this case, your feelings are an important signal to listen to.

Now, if you think your work is high expected value but the likelihood of payoff is low, it's quite common to find this dispiriting.

Humans were not evolved to be motivated by expected value calculations.

Here are some different ways to get different parts of your brain on board with a high risk high reward strategy:

Pick a strategy that you enjoy the process of. Then, even if it doesn’t work. . . at least you had a good time!

If you’re only working for the outcome and the outcome looks bleak, that's very demotivating. If you’re working for the outcome and [...]

---

Outline:

(00:38) How do you fix AI safety hopelessness?

(01:18) Pick a strategy that you enjoy the process of. Then, even if it doesn't work. . . at least you had a good time!

(01:39) Understanding the cause of your unhappiness can help you avoid certain strategies that are unlikely to work

(02:33) Ask yourself: do you feel hopeless about other things too? Or is it specific to Al safety?

(03:08) Look at the timeline of the hopelessness and your mental health.

(03:34) Set aside time each day to focus on visualizing the scenarios where your plan does work.

(04:19) Surround yourself with people who are hopeful.

(05:07) Consume inspirational biographies/biopics.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Through the MATS program, we (Alex Turner and Alex Cloud[1]) help alignment researchers grow from seeds into majestic trees. We have fun, consistently make real alignment progress, and help scholars tap into their latent abilities.

MATS summer '26 applications are open until January 18th!

Many mentees now fill impactful roles.

1. Lisa Thiergart (MATS 3.0) moved on to being a research lead at MIRI and is now a senior director at the SL5 task force.
2. Alex Cloud (MATS 6.0) went from mentee to co-mentor in one round and also secured a job at Anthropic. Lead author on the Subliminal Learning paper.
3. Jacob Goldman-Wetzler (MATS 6.0) also accepted an offer from Anthropic!
4. Luke Marks accepted work with Redwood Research after MATS 8.0.
5. And several mentees have gone on to the Anthropic Fellows program.

We likewise have a strong track record in research outputs, including

1. Pioneering steering vectors for use in LLMs (Steering GPT-2-XL by Adding an Activation Vector, Steering LLAMA-2 With Contrastive Activation Additions),
2. Masking Gradients to Localize Computation in Neural Networks,
3. Distillation [...]

---

Outline:

(03:01) Testimonials

(03:05) Lisa Thiergart

(03:47) Jacob Goldman-Wetzler

(04:14) Bruce Lee

(04:44) Ariana Azarbal

(05:32) Apply today

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Linkpost URL:
https://turntrout.com/team-shard

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: Fairly confident in the framework, uncertain about object-level claims. Keen to receive pushback on the thought experiments.

TL;DR: I argue that Whole Brain Emulations (WBEs) would clearly have moral patienthood, and that the relevant features are computational, not biological. Recent Mechanistic Interpretability (MI) work shows Large Language Models (LLMs) have emotional representations with geometric structure matching human affect. This doesn't prove LLMs deserve moral consideration, but it establishes a necessary condition, and we should take it seriously.

Acknowledgements: Thanks to Boyd Kane, Anna Soligo, and Isha Gupta for providing feedback on early drafts.

In this post I’ll be arguing for the following claim: we can make empirical progress on AI welfare without solving consciousness.

The key move is using Whole Brain Emulation as an anchor point. WBEs would clearly deserve moral consideration (under functionalism), and they're non-biological, so whatever grounds their moral status must be computational. This gives us something concrete to look for in LLMs.

In this post I'll:

1. Argue that WBEs establish a moral precedent that rules out biological prerequisites
2. Show that LLMs have human-like geometric structure in their emotional representations
3. Examine (and mostly dismiss) other candidate features that might be necessary for moral [...]

---

Outline:

(01:46) The WBE Anchor: Why Substrate Doesnt Matter

(04:18) LLMs Have Human-Like Emotional Geometry

(08:31) Ruling Out Alternative Criteria

(12:54) What Im NOT Claiming

(13:25) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The Revolution of Rising Requirements has many elements. The most onerous are the supervisory requirements on children. They have become, as Kelsey Piper recently documented, completely, utterly insane, to the point where:

Whereas I think that if you don’t allow your 10-year-old to play alone in a park, that is a much better (although still quite bad) potential reason for a CPS investigation.

This is not an idle threat, per the common statistic that around 35% of American families get investigated by CPS. Even if you are confident that will ultimately turn out fine, and given the vagaries and insanities one can never fully be sure, the process is already the punishment.

As Kelsey Piper says [...]

---

Outline:

(01:47) We Don't Let Kids Be Kids

(03:42) Let Your Children Play

(05:36) The Algorithm of Fear

(07:33) The Only Thing We Have To Fear Is Fear Itself

(11:48) No Means No

(14:20) Child Abduction Is Either Custody Disputes Or Government Intervention

(17:29) Stranger Danger Danger

(20:39) The Before Times

(21:33) Could It Be... Murder?

(22:30) The Long Road Home

(22:47) Either Bored Or Happy

(23:53) All Eyes on Me

(27:03) Screens From The Past

(28:50) Medicalization

(31:02) Television

(32:14) Suspiciously Large Effect Size

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A key risk factor for scheming (and misalignment more generally) is opaque reasoning ability.

Important caveat: To get human completion times, I ask Opus 4.5 (with thinking) to estimate how long it would take the median AIME participant to complete a given problem.

Here are the 50% reliability time horizon results:

I find that Opus 4.5 has a no-CoT 50% reliability time horizon of 3.5 minutes and that time horizon has been doubling every 9 months.

In an earlier post (Recent LLMs can leverage filler tokens or repeated problems to improve (no-CoT) math performance), I found that repeating the problem substantially boosts performance.

---

Outline:

(02:33) Some details about the time horizon fit and data

(04:17) Analysis

(06:59) Appendix: scores for Gemini 3 Pro

(11:41) Appendix: full result tables

(11:46) Time Horizon - Repetitions

(12:07) Time Horizon - Filler

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[This is an entry for lsusr's write-like-lsusr competition.]

"I solved the alignment problem," said Qianyi.

"You what?" said postdoc Timothy.

It was late at the university computer laboratory and Timothy' skepticism was outvoted by his eagerness to think about anything other than his dissertation.

"You heard me," said Qianyi.

"You do realize that solving the alignment has lots of different components, right?" said Timothy, "First you need to figure out how to build a general superintelligence and world optimizer."

"I did that," said Qianyi.

"Then you'd need to align it with the Coherent Extrapolated Volition (CEV) of humanity," said Timothy.

"I did that too," said Quanyi.

"Except CEV is barely even a coherent concept. This isn't even a technical problem. It's a socio-ontological one. If people disagree with each other, then CEV is undefined. And every large group of people on Planet Earth has holdouts who disagree about everything you can imagine. There are subcultures who believe in underground lizardpeople," said Timothy.

"Solved it."

"There's also the problem of falsified preferences. What people say they want and what people actually want. What people say they believe differs from [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Merry Christmas! Today I turn an earlier LW shortform into a full post and discuss "unknown knowns" "obvious" ideas that are actually hard to discuss because they're invisible when you don't have them, and then almost impossible to unsee when you do.

Hopefully this is a fun article for like the twenty people who check LW on Christmas!

__

There are a number of implicit concepts I have in my head that seem so obvious that I don’t even bother verbalizing them. At least, until it's brought to my attention other people don’t share these concepts.

It didn’t feel like a big revelation at the time I learned the concept, just a formalization of something that's extremely obvious. And yet other people don’t have those intuitions, so perhaps this is pretty non-obvious in reality.

Here's a short, non-exhaustive list:

• Intermediate Value Theorem
• Net Present Value
• Differentiable functions are locally linear
• Grice's maxims
• Theory of Mind

If you have not heard any of these ideas before, I highly recommend you look them up! Most *likely*, they will seem obvious to you. You might already know those concepts by a different name, or they’re already integrated enough into your worldview [...]

---

Outline:

(02:04) The Intermediate Value Theorem

(03:54) Net Present Value

(05:24) Differentiable functions are locally linear

(07:01) Grice's maxims

(08:17) The Pig Hates It

(08:20) Theory of Mind

(09:58) Why this all matters

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

…if it's anything like mine[1]. If it isn't, then maybe treat this as an opportunity to fight the typical mind fallacy by getting a glimpse at what's going on in other minds.

I've made many predictions and have gone through a lot of calibration training over the years, and have generally become pretty well calibrated in several domains. Yet I noticed that in some areas, I seem surprisingly immune to calibration. Where my brain just doesn't want to learn that it's systematically and repeatedly wrong about something. This post is about the three four such areas.

(1) Future Ability to Remember Things

Sometimes, I feel incredibly confident that some piece of information will be easy for me to recall in the future, even when this is entirely false. Some examples:

• When I meet a new person, and they mention their name, then while hearing it, I typically feel like "yeah, easy, it's trivial for me to repeat that name in my head so I will remember it forever, no problem", and then 5 seconds later it's gone
• When I open some food/drink product and then put it into the fridge, I usually take a note on the product [...]

---

Outline:

(00:43) (1) Future Ability to Remember Things

(02:32) (2) Local Optima of Comfort

(04:13) (3) Interpersonal Conflict

(05:26) (4) Bonus: Recognizing Im in a Dream

(06:19) Takeaway

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.