Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

TL;DR: In this post, I distinguish between two related concepts in neural network interpretability: polysemanticity and superposition. Neuron polysemanticity is the observed phenomena that many neurons seem to fire (have large, positive activations) on multiple unrelated concepts. Superposition is a specific explanation for neuron (or attention head) polysemanticity, where a neural network represents more sparse features than there are neurons (or number of/dimension of attention heads) in near-orthogonal directions. I provide three ways neurons/attention heads can be polysemantic without superposition: non--neuron aligned orthogonal features, non-linear feature representations, and compositional representation without features. I conclude by listing a few reasons why it might be important to distinguish the two concepts.

Epistemic status: I wrote this “quickly” in about 10 hours, as otherwise it wouldn’t have come out at all. Think of it as a (failed) experiment in writing [...]

---

Outline:

(04:23) A brief review of polysemanticity and superposition

(04:28) Neuron polysemanticity

(08:01) Superposition

(12:17) Polysemanticity without superposition

(12:32) Example 1: non–neuron aligned orthogonal features

(17:25) Example 2: non-linear feature representations

(19:01) Example 3: compositional representation without “features”

(20:35) Conclusion: why does this distinction matter?

(21:38) Our current model of superposition may not fully explain neuron polysemanticity, so we should keep other hypotheses in mind

(23:53) Attempts to “solve superposition” may actually only be solving easier cases of polysemanticity

(25:02) Clear definitions are important for clear communication and rigorous science

(25:47) Acknowledgements

The original text contained 24 footnotes which were omitted from this narration.

---

First published:

April 26th, 2024

Source:

https://www.lesswrong.com/posts/8EyCQKuWo6swZpagS/superposition-is-not-just-neuron-polysemanticity

---

Narrated by TYPE III AUDIO.

This is a linkpost for On Duct Tape and Fence Posts.

Eliezer writes about fence post security. When people think to themselves "in the current system, what's the weakest point?", and then dedicate their resources to shoring up the defenses at that point, not realizing that after the first small improvement in that area, there's likely now a new weakest point somewhere else.

Fence post security happens preemptively, when the designers of the system fixate on the most salient aspect(s) and don't consider the rest of the system. But this sort of fixation can also happen in retrospect, in which case it manifest a little differently but has similarly deleterious effects.

Consider a car that starts shaking whenever it's driven. It's uncomfortable, so the owner gets a pillow to put on the seat. Items start falling off the dash, so they get a tray to put them [...]

---

First published:

April 26th, 2024

Source:

https://www.lesswrong.com/posts/CLS4FijfEFHzc5HEv/duct-tape-security

---

Narrated by TYPE III AUDIO.

Epistemic Status: Musing and speculation, but I think there's a real thing here.

1.

When I was a kid, a friend of mine had a tree fort. If you've never seen such a fort, imagine a series of wooden boards secured to a tree, creating a platform about fifteen feet off the ground where you can sit or stand and walk around the tree. This one had a rope ladder we used to get up and down, a length of knotted rope that was tied to the tree at the top and dangled over the edge so that it reached the ground.

Once you were up in the fort, you could pull the ladder up behind you. It was much, much harder to get into the fort without the ladder. Not only would you need to climb the tree itself instead of the ladder with its handholds, but [...]

---

Outline:

(00:11) 1.

(01:33) 2.

(05:03) 3.

(08:45) 4.

(11:40) 5.

The original text contained 1 footnote which was omitted from this narration.

---

First published:

April 26th, 2024

Source:

https://www.lesswrong.com/posts/k2kzawX5L3Z7aGbov/on-not-pulling-the-ladder-up-behind-you

---

Narrated by TYPE III AUDIO.

My credence: 33% confidence in the claim that the growth in the number of GPUs used for training SOTA AI will slow down significantly directly after GPT-5. It is not higher because of (1) decentralized training is possible, and (2) GPT-5 may be able to increase hardware efficiency significantly, (3) GPT-5 may be smaller than assumed in this post, (4) race dynamics.

TLDR: Because of a bottleneck in energy access to data centers and the need to build OOM larger data centers.

The reasoning behind the claim:

• Current large data centers consume around 100 MW of power, while a single nuclear power plant generates 1GW. The largest seems to consume 150 MW.
• An A100 GPU uses 250W, and around 1kW with overheard. B200 GPUs, uses ~1kW without overhead. Thus a 1MW data center can support maximum 1k to 2k GPUs.
• GPT-4 used something like 15k to 25k GPUs [...]

---

Outline:

(00:42) The reasoning behind the claim:

(03:18) Unrelated to the claim:

(04:23) How big is that effect going to be?

(06:52) Impact of GPT-5

---

First published:

April 26th, 2024

Source:

https://www.lesswrong.com/posts/xyL5kb8RBGLiupGLf/scaling-of-ai-training-runs-will-slow-down-after-gpt-5

---

Narrated by TYPE III AUDIO.

(Half-baked work-in-progress. There might be a “version 2” of this post at some point, with fewer mistakes, and more neuroscience details, and nice illustrations and pedagogy etc. But it's fun to chat and see if anyone has thoughts.)

1. Background

There's a neuroscience problem that's had me stumped since almost the very beginning of when I became interested in neuroscience at all (as a lens into AGI safety) back in 2019. But I think I might finally have “a foot in the door” towards a solution!

What is this problem? As described in my post Symbol Grounding and Human Social Instincts, I believe the following:

• (1) We can divide the brain into a “Learning Subsystem” (cortex, striatum, amygdala, cerebellum and a few other areas) on the one hand, and a “Steering Subsystem” (mostly hypothalamus and brainstem) on the other hand; and a human's “innate drives” (roughly equivalent to [...]

---

Outline:

(00:22) 1. Background

(04:22) 2. New stuff

(04:25) 2.1 Ingredient 1: “Spatial attention”

(04:31) 2.1.1 What is “spatial attention” intuitively?

(05:38) 2.1.2 How does spatial attention work in the brain?

(08:37) 2.2 Empathetic simulation version 1

(11:35) 2.3 Ingredient 2: A brainstem “is-a-person” flag

(13:21) 2.4 Empathetic simulation version 2

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

April 26th, 2024

Source:

https://www.lesswrong.com/posts/7Pt9fogptmiSduXt9/spatial-attention-as-a-tell-for-empathetic-simulation

---

Narrated by TYPE III AUDIO.

Crosspost from my blog.

If you spend a lot of time in the blogosphere, you’ll find a great deal of people expressing contrarian views. If you hang out in the circles that I do, you’ll probably have heard of Yudkowsky say that dieting doesn’t really work, Guzey say that sleep is overrated, Hanson argue that medicine doesn’t improve health, various people argue for the lab leak, others argue for hereditarianism, Caplan argue that mental illness is mostly just aberrant preferences and education doesn’t work, and various other people expressing contrarian views. Often, very smart people—like Robin Hanson—will write long posts defending these views, other people will have criticisms, and it will all be such a tangled mess that you don’t really know what to think about them.

For a while, I took a lot of these contrarian views pretty seriously. If I’d had to bet 6-months ago, I’d [...]

---

First published:

April 25th, 2024

Source:

https://www.lesswrong.com/posts/qBDQQqQ9dWhJJ7Jt6/losing-faith-in-contrarianism

---

Narrated by TYPE III AUDIO.

This is a linkpost for https://justismills.substack.com/p/llms-seem-relatively-safe

Post for a somewhat more general audience than the modal LessWrong reader, but gets at my actual thoughts on the topic.

In 2018 OpenAI defeated the world champions of Dota 2, a major esports game. This was hot on the heels of DeepMind's AlphaGo performance against Lee Sedol in 2016, achieving superhuman Go performance way before anyone thought that might happen. AI benchmarks were being cleared at a pace which felt breathtaking at the time, papers were proudly published, and ML tools like Tensorflow (released in 2015) were coming online. To people already interested in AI, it was an exciting era. To everyone else, the world was unchanged.

Now Saturday Night Live sketches use sober discussions of AI risk as the backdrop for their actual jokes, there are hundreds of AI bills moving through the world's legislatures, and Eliezer Yudkowsky is featured in Time [...]

---

Outline:

(01:44) LLMs are self limiting

(05:08) LLMs are decent at human values

(09:03) Playing human roles is pretty human

(10:45) And So

---

First published:

April 25th, 2024

Source:

https://www.lesswrong.com/posts/ZxAWeiT8qNYppPbYA/llms-seem-relatively-safe

Linkpost URL:
https://justismills.substack.com/p/llms-seem-relatively-safe

---

Narrated by TYPE III AUDIO.

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a linkpost for https://arxiv.org/abs/2404.16014

Authors: Senthooran Rajamanoharan*, Arthur Conmy*, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, Neel Nanda

A new paper from the Google DeepMind mech interp team: Improving Dictionary Learning with Gated Sparse Autoencoders!

Gated SAEs are a new Sparse Autoencoder architecture that seems to be a significant Pareto-improvement over normal SAEs, verified on models up to Gemma 7B. They are now our team's preferred way to train sparse autoencoders, and we'd love to see them adopted by the community! (Or to be convinced that it would be a bad idea for them to be adopted by the community!)

They achieve similar reconstruction with about half as many firing features, and while being either comparably or more interpretable (confidence interval for the increase is 0%-13%).

See Sen's Twitter summary, my Twitter [...]

---

First published:

April 25th, 2024

Source:

https://www.lesswrong.com/posts/vdoiWXeouGsZEYgrg/improving-dictionary-learning-with-gated-sparse-autoencoders

Linkpost URL:
https://arxiv.org/abs/2404.16014

---

Narrated by TYPE III AUDIO.

This is a linkpost for https://www.wsj.com/business/retail/amazon-secret-operation-intel-rivals-eb82ea3c

Staff went undercover on Walmart, eBay and other marketplaces as a third-party seller called ‘Big River.’ The mission: to scoop up information on pricing, logistics and other business practices.

The operation, called Big River Services International, sells around $1 million a year of goods through e-commerce marketplaces including eBay, Shopify, Walmart and Amazon AMZN 1.49%increase; green up pointing triangle.com under brand names such as Rapid Cascade and Svea Bliss. “We are entrepreneurs, thinkers, marketers and creators,” Big River says on its website. “We have a passion for customers and aren’t afraid to experiment.”

What the website doesn’t say is that Big River is an arm of Amazon that surreptitiously gathers intelligence on the tech giant's competitors.

Born out of a 2015 plan code named “Project Curiosity,” Big River uses its sales across multiple countries to obtain pricing data, logistics information and other [...]

---

First published:

April 23rd, 2024

Source:

https://www.lesswrong.com/posts/eDaPLeLcEftrGSopD/wsj-inside-amazon-s-secret-operation-to-gather-intel-on

Linkpost URL:
https://www.wsj.com/business/retail/amazon-secret-operation-intel-rivals-eb82ea3c

---

Narrated by TYPE III AUDIO.

This is a linkpost for https://www.orwellfoundation.com/the-orwell-foundation/orwell/essays-and-other-works/why-i-write/

People have been posting great essays so that they're "fed through the standard LessWrong algorithm." This essay is in the public domain in the UK but not the US.

From a very early age, perhaps the age of five or six, I knew that when I grew up I should be a writer. Between the ages of about seventeen and twenty-four I tried to abandon this idea, but I did so with the consciousness that I was outraging my true nature and that sooner or later I should have to settle down and write books.

I was the middle child of three, but there was a gap of five years on either side, and I barely saw my father before I was eight. For this and other reasons I was somewhat lonely, and I soon developed disagreeable mannerisms which made me unpopular throughout my [...]

---

First published:

April 25th, 2024

Source:

https://www.lesswrong.com/posts/Li4evP8nL7Xg4Wjjw/why-i-write-by-george-orwell-1946

Linkpost URL:
https://www.orwellfoundation.com/the-orwell-foundation/orwell/essays-and-other-works/why-i-write/

---

Narrated by TYPE III AUDIO.