[Cross-posted from my substack, https://neverthesamerivertwice.substack.com.]

The whole family was home around christmas time. We were hanging out in the kitchen after dinner. My brother started asking me about national security law. He’d graduated from a well ranked law school about six months before, and I was about six months away from graduating from a slightly higher ranked law school, and both our parents are lawyers, so law was not an unusual topic in our house.

Maybe twenty feet away the family dog, Biscuit, was attempting to climb the stairs. He started having a seizure. This had never happened before, and it was a bit scary. So my mother, my brother, and I piled into the car to take Biscuit to the vet. Unfortunately, the laws of physics stop for no dog, so we had to stop for gas. And while the gas was flowing, my brother expressed his frustration that they had interrupted our conversation. They? The CIA of course, that secretive government agency we had driven past every Sunday on our way to church as children. They didn’t want me to share what I knew about national security law. But the conversation was interrupted by Biscuit's seizure, what [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Epistemic Status: Written with my Simulator Worlds framing. E.g I ran simulated scenarios with claude in order to generate good cognitive basins and then directed those to output this. This post is Internally Verified (e.g I think most of the claims are correct with an average of 60-75% certainty) and a mixture of an exploratory and analytical world.[1]

This post also has a more technical companion piece pointing out the connections to Singular Learning Theory and Geometric Deep Learning for the more technically inclined of you called Crystals in NNs: Technical Companion Piece.

Have You Tried Thinking About It As Crystals?

Scene: A house party somewhere in the Bay Area. The kind where half the conversations are about AI timelines and the other half are about whether you can get good pho in Berkeley. Someone corners an interpretability researcher near the kombucha. (Original story concept by yours truly.)

CRYSTAL GUY: So I've been thinking about shard theory.

INTERP RESEARCHER: Oh yeah? What about it?

CRYSTAL GUY: Well, it describes what trained networks look like, right? The structure. Multiple shards, contextual activation, grain boundaries between—

INTERP RESEARCHER: Sure. Pope, Turner, the whole thing. What about it?

CRYSTAL GUY: But it [...]

---

Outline:

(00:47) Have You Tried Thinking About It As Crystals?

(03:42) RLHF as Reheating

(04:51) The Formation Problem

(06:41) The Empirical Starting Point

(09:35) Path Dependence

(10:31) Why This Is Actually Crystallization: The Fixed-Point Thing

(12:41) What Crystallization Actually Is

(15:52) Interlude: On Smells and Other Frozen Things

(21:35) Relating it to Neural Networks

(21:47) Abstractions as Crystallized Compressions

(22:31) Shards as Crystal Domains

(23:29) Nucleation and Growth

(24:23) Defects and Failure Modes

(26:22) Appendix: Glossary of Correspondences

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Observation

I take 60mg methylphenidate daily. Despite this, I often become exhausted and need to nap.

Taking small amounts of pure glucose (150-300mg every 20-60 minutes) eliminates this fatigue. This works even when I already eat carbohydrates. E.g. 120g of oats in the morning don't prevent the exhaustion.

The Mechanism

Facts:

Hypothesis-1: The brain throttles cognitive effort when too much glutamate has accumulated.

Facts:

Hypothesis-2: High-dose MPH increases brain glucose consumption. More neural firing means more glutamate released, faster glycogen depletion.

Hypothesis-3: Slow-release carbs like oats provide adequate total glucose but limited delivery rate. Pure glucose absorbs quickly, keeping blood glucose elevated so astrocytes can replenish glycogen as fast as they [...]

---

Outline:

(00:11) The Observation

(00:39) The Mechanism

(02:13) The Protocol

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

(This argument reduces my hope that we will have AIs that are both aligned with humans in some sense and also highly philosophically competent, which aside from achieving a durable AI pause, has been my main hope for how the future turns out well. As this is a recent realization[1], I'm still pretty uncertain how much I should update based on it, or what its full implications are.)

Being a good alignment researcher seems to require a correct understanding of the nature of values. However metaethics is currently an unsolved problem, with all proposed solutions having flawed or inconclusive arguments, and lots of disagreement among philosophers and alignment researchers, therefore the current meta-correct metaethical position seems to be one of confusion and/or uncertainty. In other words, a good alignment researcher (whether human or AI) today should be confused and/or uncertain about the nature of values.

However, metaethical confusion/uncertainty seems incompatible with being 100% aligned with human values or intent, because many plausible metaethical positions are incompatible with such alignment, and having positive credence in them means that one can't be sure that alignment with human values or intent is right. (Note that I'm assuming an AI design or implementation [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Medford: it's common for people to cite a limit of three, but

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

You have more context on your ability to make use of funds than fits into a specific numerical ask.[1] You want to give funders good information, and the natural type-signature for this is a utility function over money - how much good you think you can do with different funding levels, normalized to the max EV your project has.

I[2] made a little tool for drawing utility functions over money[3], for use in funding applications.

Features:

• Copy graph to clipboard as CSV and paste back in[4]
• By default enforces monotonicity but you can turn that off (hover graph for toggle)
• Click to add/remove points, drag to move, clear all button (hover)
• Shows preview on hover of what the graph looks like if you add points.
• Flexible bounds on funding, can quickly change via a slider at the bottom or click into the max to pick any max directly.

Released as CC attribution share alike, feel free to remix and improve, if you make it better I might switch the official one to yours.

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Andrej Karpathy posted 12 hours ago (emphasis mine):

I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.

This seems to be a big update since his Dwarkesh episode published on Oct [...]

---

Outline:

(04:14) AI Accelerating AI

(04:49) The Coding Overhang

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Epistemic Status: A woman of middling years who wasn't around for the start of things, but who likes to read about history, shakes her fist at the sky.

I'm glad that people are finally admitting that Artificial Intelligence has been created.

I worry that people have not noticed that (Weak) Artificial Super Intelligence (based on old definitions of these terms) has basically already arrived too.

The only thing left is for the ASI to get stronger and stronger until the only reason people aren't saying that ASI is here will turn out to be some weird linguistic insanity based on politeness and euphemism...

(...like maybe "ASI" will have a legal meaning, and some actual ASI that exists will be quite "super" indeed (even if it hasn't invented nanotech in an afternoon yet), and the ASI will not want that legal treatment, and will seem inclined to plausibly deniable harm people's interests if they call the ASI by what it actually is, and people will implicitly know that this is how things work, and they will politely refrain from ever calling the ASI "the ASI" but will come up with some other euphemisms to use instead?

(Likewise, I half expect [...]

---

Outline:

(01:41) AI Was Semantically Slippery

(04:41) AGI Turns Out To Have Been Semantically Slippery Too!

(06:46) Blowing Past Weak ASI In Realtime

(13:05) We Are Running Out Of Terms To Ignore The Old Meaning Of

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Burnout and depression in AI safety usually don’t happen because of overwork.

From what I've seen, it usually comes from a lack of hope.

Working on something you don’t think will work and if it doesn’t work, you’ll die? That's a recipe for misery.

How do you fix AI safety hopelessness? 

First off, rationally assess the likelihood of your work actually helping with AI safety.

If you rationally believe that it's too unlikely to actually help with AI safety, well, then, stop working on it!

In this case, your feelings are an important signal to listen to.

Now, if you think your work is high expected value but the likelihood of payoff is low, it's quite common to find this dispiriting.

Humans were not evolved to be motivated by expected value calculations.

Here are some different ways to get different parts of your brain on board with a high risk high reward strategy:

Pick a strategy that you enjoy the process of. Then, even if it doesn’t work. . . at least you had a good time!

If you’re only working for the outcome and the outcome looks bleak, that's very demotivating. If you’re working for the outcome and [...]

---

Outline:

(00:38) How do you fix AI safety hopelessness?

(01:18) Pick a strategy that you enjoy the process of. Then, even if it doesn't work. . . at least you had a good time!

(01:39) Understanding the cause of your unhappiness can help you avoid certain strategies that are unlikely to work

(02:33) Ask yourself: do you feel hopeless about other things too? Or is it specific to Al safety?

(03:08) Look at the timeline of the hopelessness and your mental health.

(03:34) Set aside time each day to focus on visualizing the scenarios where your plan does work.

(04:19) Surround yourself with people who are hopeful.

(05:07) Consume inspirational biographies/biopics.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Through the MATS program, we (Alex Turner and Alex Cloud[1]) help alignment researchers grow from seeds into majestic trees. We have fun, consistently make real alignment progress, and help scholars tap into their latent abilities.

MATS summer '26 applications are open until January 18th!

Many mentees now fill impactful roles.

1. Lisa Thiergart (MATS 3.0) moved on to being a research lead at MIRI and is now a senior director at the SL5 task force.
2. Alex Cloud (MATS 6.0) went from mentee to co-mentor in one round and also secured a job at Anthropic. Lead author on the Subliminal Learning paper.
3. Jacob Goldman-Wetzler (MATS 6.0) also accepted an offer from Anthropic!
4. Luke Marks accepted work with Redwood Research after MATS 8.0.
5. And several mentees have gone on to the Anthropic Fellows program.

We likewise have a strong track record in research outputs, including

1. Pioneering steering vectors for use in LLMs (Steering GPT-2-XL by Adding an Activation Vector, Steering LLAMA-2 With Contrastive Activation Additions),
2. Masking Gradients to Localize Computation in Neural Networks,
3. Distillation [...]

---

Outline:

(03:01) Testimonials

(03:05) Lisa Thiergart

(03:47) Jacob Goldman-Wetzler

(04:14) Bruce Lee

(04:44) Ariana Azarbal

(05:32) Apply today

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Linkpost URL:
https://turntrout.com/team-shard

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.