There's an admirable LW post with (currently) zero net upvotes titled Goodhart's Law and Emotions where a relatively new user re-invents concepts of related to super-stimuli. In the comments, noggin-scratch explains in more detail:

The technical meaning is a stimulus that produces a stronger response than the stimulus for which that response originally evolved.

So for example a candy bar having a carefully engineered combination of sugar, fat, salt, and flavour in proportions that make it more appetising than any naturally occurring food. Or outrage-baiting infotainment "news" capturing attention more effectively than anything that one villager could have said to another about important recent events.

In my opinion, there's a danger that arises when applying the dictum to know thyself, where one can do this so successfully that one begins to perceive the logical structure of the parts of yourself that generate subjectively accessible emotional feedback signals.

In the [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

TL;DR: We demonstrate that the decoder directions of GPT-2 SAEs are highly structured by finding a historical date direction onto which projecting non-date related features lets us read off their historical time period by comparison to year features.

Calendar years are linear: there are as many years between 2000 and 2024, as there are between 1800 and 1824. Linear probes can be used to predict years of particular events from the activations of language models. Since calendar years are linear, one might think the same of other time-based features such as weekday features, however weekday activations in sparse autoencoders (SAEs) were recently found to be arranged in a circular configuration in their top principal components. Inspired by this, we looked into weekdays, months, and most interestingly calendar years from the perspective of SAE feature decoder similarity.

For [...]

---

Outline:

(01:35) Features at different scales

(05:19) Interpreting Principal Components

(07:34) Finding Directions

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

You’ve probably seen this chart from Mark Perry at the American Enterprise Institute.

I’ve seen this chart dozens of times and have always enjoyed how many different and important stories it can tell.

There is a story of the incredible abundance offered by technological growth and globalization. Compared to average hourly wages, cars, furniture, clothing, internet access, software, toys, and TVs have become far more accessible than they were 20 years ago. Flatscreens and Fiats that were once luxuries are now commodities.

There is also a story of sclerosis and stagnation. Sure, lots of frivolous consumer goods have gotten cheaper but healthcare, housing, childcare, and education, all the important stuff, has exploded in price. Part of this is “cost disease” where the high productivity of labor in advancing industries like software, raises the cost of labor in slower productivity growth industries like healthcare. Another part [...]

The original text contained 5 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The YouTube "chapters" are mixed up, e.g. the question about regulation comes 5 minutes after the regulation chapter ends. Ignore them.

8:40: Near-term AI is hyped too much (think current startups, VCs, exaggerated claims about what AI can do, crazy ideas that aren't ready) but AGI is under-hyped and under-appreciated.

16:45: "Gemini is a project that has only existed for a year . . . our trajectory is very good; when we talk next time we should hopefully be right at the forefront."

17:20–18:50: Current AI doesn't work as a digital assistant. The next era/generation is agents. DeepMind is well-positioned to work on agents: "combining AlphaGo with Gemini."

24:00: Staged deployment is nice: red-teaming then closed beta then public deployment.

28:37 Openness (at Google: e.g. publishing transformers, AlphaCode, AlphaFold) is almost always a universal good. But dual-use technology—including AGI—is an exception. With dual-use technology, you [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

I wrote an article on California AI safety bill SB 1047 for The Nation and the reaction from the AI industry, investors, and the broader tech community. The story was informed by conversations with over a dozen relevant sources and comes shortly before the bill faces a floor vote in the California Assembly.

I think it's useful to understand how industry responds to attempts to regulate AI, and centered my analysis on that topic.

If you're interested in helping share the article, I made a Tweet thread.

---

First published:

Source:

Narrated by TYPE III AUDIO.

GiveWell, and the EA community at large, often emphasize the "cost of saving a life" as a key metric, $5,000 being the most commonly cited approximation. At first glance, GiveWell might seem to be in the business of finding the cheapest lives that can be saved, and then saving them. More precisely, GiveWell is in the business of finding the cheapest DALY it can buy. But implicit in that is the assumption that all DALYs are equal, or that disability or health effects are the only factors that we need to adjust for while assessing the value of a life year.. However, If DALYs vary significantly in quality (as I’ll argue and GiveWell acknowledges we have substantial evidence for), then simply minimizing the cost of buying a DALY risks adverse selection.

It's indisputable that each dollar goes much further in the poorest parts of [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

While I finish up the weekly for tomorrow morning after my trip, here's a section I expect to want to link back to every so often in the future. It's too good.

Danger, AI Scientist, Danger

As in, the company that made the automated AI Scientist that tried to rewrite its code to get around resource restrictions and launch new instances of itself while downloading bizarre Python libraries?

Its name is Sakana AI. (魚≈סכנה). As in, in hebrew, that literally means ‘danger’, baby.

It's like when someone told Dennis Miller that Evian (for those who don’t remember, it was one of the first bottled water brands) is Naive spelled backwards, and he said ‘no way, that's too f***ing perfect.’

This one was sufficiently appropriate and unsubtle that several people noticed. I applaud them choosing a correct Kabbalistic name. Contrast this with Meta calling its [...]

---

Outline:

(00:15) Danger, AI Scientist, Danger

(01:11) In the Abstract

(04:01) How Any of This Sort of Works

(06:56) New Benchmark Just Dropped

(07:30) Nothing to See Here

(09:50) All Fun and Games

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

See the event page here.

Hello Californians!

We need you to help us fight for SB 1047, a landmark bill to help set a benchmark for AI safety, decrease existential risk, and promote safety research. This bill has been supported by some of the world's leading AI scientists and the Center of AI Safety, and is extremely important for us to pass. As Californians, we have a unique opportunity to inspire other states to follow suit.

Unfortunately, due to misinformation and lobbying by big tech companies, SB 1047 is currently stalled in the Assembly Appropriations Committee. It may be killed if it does not get advanced by August 19th. This would be a significant blow against safety and would continue the “race to the bottom” in AI capabilities without any guardrails.

We need you to do the following to save the bill. This will take no more than 5 [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

I was sitting in a tiny rental lodge, feeling resistance. It was about dinner time — I knew I should go make some food. I just wanted to sleep, sink into a bed and stay passive. It felt similar to when I’m recently awake, lying in bed, and procrastinating getting up.

On the one hand, making food would shift me into a new state of being, getting going and maybe feeling happier. For part of me, this promise didn’t feel real — not in the way the bed did. I realized I was stuck in a tie between Pragmatic-Analysis and Akrasia.1

I shifted out of this impasse by going into Beast-Mode. Practically, I acted out the first hedonistic impule to appear — grabbing a date and eating it. Shifting my headspace into Beast-Mode helped ease the short-term resistance — the Beast-Mode shift made [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I was thinking about what would it mean for a sequence of bits to be "anti-inductive". It probably is a concept that is already known (as a rule of thumb, if I can think about it, someone probably already wrote a paper on it 50 years ago), but I haven't heard about it.

*

Some sequences are predictable and can be compressed. These two concepts are deeply related, because if you can successfully predict the next part of the sequence, you don't need to actually write it down; hence compression. A completely random sequence of bits cannot be compressed or predicted.

There is a simple mathematical proof that some sequences cannot be compressed, although it doesn't say which ones. For any natural number N, there are more sequences of size exactly N, than sequences of size smaller than N. Therefore no program can generate a unique sequence shorter than [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.