It was a dark and stormy night.

The prospect held the front of his cloak tight to his chest. He stumbled, fell over into the mud, and picked himself back up. Shivering, he slammed his body against the front doors of the Temple and collapsed under its awning.

He picked himself up and slammed his fists against the double ironwood doors. He couldn't hear his own knocks above the gale. He banged harder, then with all his strength.

"Hello! Is anyone in there? Does anyone still tend the Fire?" he implored.

There was no answer.

The Temple's stone walls were built to last, but rotting plywood covered the apertures that once framed stained glass. The prospect slumped down again, leaning his back against the ironwood. He listened to the pitter-patter of rain on overgrowth. It wasn't a bad place to think.

The trouble [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

AI safety researchers often rely on LLM “judges” to qualitatively evaluate the output of separate LLMs. We try this for our own interpretability research, but find that our LLM judges are often deeply biased. For example, we use Llama2 to judge whether movie reviews are more “(A) positive” or “(B) negative”, and find that it almost always answers “(B)”, even when we switch the labels or order of these alternatives. This bias is particularly surprising for two reasons: first, because we expect a fairly capable model like Llama2 to perform well at a simple sentiment classification task like this, and second, because this specific “(B)”-bias doesn’t map on to a human bias we’d expect to see in the training data. We describe our experiments, provide code to replicate our results, and offer suggestions to mitigate [...]

---

Outline:

(03:25) Sentiment classification task

(05:30) Llama2's (B)-bias

(09:34) What can we do?

(12:20) Conclusion

The original text contained 10 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is a D&D.Sci scenario: a puzzle where players are given a dataset to analyze and an objective to pursue using information from that dataset.

You steel your nerves as the Mad Tyrant[1] peers at you from his throne. In theory, you have nothing to worry about: since the Ninety Degree Revolution last year, His Malevolence[2] has had his power sharply curtailed, and his bizarre and capricious behavior has shifted from homicidally vicious to merely annoying. So while everyone agrees he's still getting the hang of this whole “Constitutional Despotism”[3] thing, and while he did drag you before him in irons when he heard a Data Scientist was traveling through his territory, you’re still reasonably confident you’ll be leaving with all your limbs attached (probably even to the same parts of your torso).

Your voice wavering only slightly, you politely inquire as to why you were summoned.

He tells [...]

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Sparse Autoencoder (SAE) errors are empirically pathological: when a reconstructed activation vector is distance _epsilon_ from the original activation vector, substituting a randomly chosen point at the same distance changes the next token prediction probabilities significantly less than substituting the SAE reconstruction[1] (measured by both KL and loss). This is true for all layers of the model (~2x to ~4.5x increase in KL and loss over baseline) and is not caused by feature suppression/shrinkage. Assuming others replicate, these results suggest the proxy reconstruction objective is behaving pathologically. I am not sure why these errors occur but expect understanding this gap will give us deeper insight into SAEs while also providing an additional metric to guide methodological progress.

As the interpretability community allocates more resources and increases reliance on SAEs, it is important to understand the limitation and potential flaws of this method.

SAEs are designed [...]

---

Outline:

(03:59) Intuition: how big a deal is this (KL) difference?

(04:30) Experiments and Results

(04:56) Intervention Types

(07:35) Layerwise Intervention Results in More Detail

(08:56) Single Token Intervention Results

(10:10) How pathological are the errors?

(11:09) When do these errors happen?

(12:26) Replication with Attention SAEs

(13:18) Concluding Thoughts

(13:22) Why is this happening?

(16:21) Takeaways

(17:15) Future work

(17:43) Acknowledgements

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This article contains more than 100 uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Lots of people already know about ACX/SSC, but I think that crossposting to LW is unusually valuable in this particular case, since lots of people were waiting for a big schelling-point overview of the 15-hour Rootclaim Lab Leak debate, and unlike LW, ACX's comment section is a massive vote-less swamp that lags the entire page.

Here are the first 10 paragraphs:

Saar Wilf is an ex-Israeli entrepreneur. Since 2016, he's been developing a new form of reasoning, meant to transcend normal human bias.

His method - called Rootclaim - uses Bayesian reasoning, a branch of math that explains the right way to weigh evidence. This isn’t exactly new. Everyone supports Bayesian reasoning. The statisticians support it, I support it, Nate Silver wrote a whole book supporting it.

But the joke goes that you do Bayesian reasoning by doing normal reasoning while muttering “Bayes, Bayes [...]

---

First published:

Source:

Linkpost URL:
https://www.astralcodexten.com/p/practically-a-book-review-rootclaim

Narrated by TYPE III AUDIO.

There's been a lot of discussion among safety-concerned people about whether it was bad for Anthropic to release Claude-3. I felt like I didn’t have a great picture of all the considerations here, and I felt that people were conflating many different types of arguments for why it might be bad. So I decided to try to write down an at-least-slightly-self-contained description of my overall views and reasoning here.

Tabooing “Race Dynamics”

I’ve heard a lot of people say that this “is bad for race dynamics”. I think that this conflates a couple of different mechanisms by which releasing Claude-3 might have been bad.

So, taboo-ing “race dynamics”, a common narrative behind these words is

As companies release better & better models, this incentivizes other companies to pursue more capable models at the expense of safety. Eventually, one company goes too far, produces unaligned AGI, and we all die”.

---

Outline:

(00:28) Tabooing “Race Dynamics”

(01:28) Did releasing Claude-3 cause other AI labs to invest less in evals/redteaming models before deployment?

(02:43) Did releasing Claude-3 divert resources away from alignment research and into capabilities research?

(03:48) Releasing Very SOTA Models

(04:18) Anthropic at the Frontier is Good?

(06:18) What I Currently Think

(06:59) Conclusion

(08:34) Appendix:

(08:38) Capabilities Leakage

(12:37) Is Increasing Capabilities Bad?

---

First published:

Source:

Narrated by TYPE III AUDIO.

TLDR

Manifold is hosting a festival for prediction markets: Manifest 2024! We’ll have serious talks, attendee-run workshops, and fun side events over the weekend. Chat with special guests like Nate Silver, Scott Alexander, Robin Hanson, Dwarkesh Patel, Cate Hall, and more at this second in-person gathering of the forecasting & prediction market community!

Tickets & more info: manifest.is

WHEN: June 7-9, 2024, with LessOnline and Summer Camp starting May 31

WHERE: Lighthaven, Berkeley, CA

WHO: Hundreds of folks, interested in forecasting, rationality, EA, economics, journalism, tech and more. If you’re reading this, you’re invited!

People

Manifest is an event for the forecasting & prediction market community, and everyone else who's interested. We’re aiming for about 500-700 attendees (you can check the markets here!).

Current speakers & special guests include:

Content

Everything's optional, and there’ll always be a bunch of sessions running concurrently. [...]

---

Outline:

(00:08) TLDR

(01:07) People

(01:28) Content

(02:35) Last year's content

(04:08) Testimonials Festimonials

(05:14) Place

(05:42) Extra events

(06:40) LessOnline

(07:19) Summer Camp

(08:28) Tickets and contact

---

First published:

Source:

Linkpost URL:
https://news.manifold.markets/p/come-to-manifest-2024-june-7-9-in

Narrated by TYPE III AUDIO.

About 15 years ago, I read Malcolm Gladwell's Outliers. He profiled Chris Langan, an extremely high-IQ person, claiming that he had only mediocre accomplishments despite his high IQ. Chris Langan's theory of everything, the Cognitive Theoretic Model of the Universe, was mentioned. I considered that it might be worth checking out someday.

Well, someday has happened, and I looked into CTMU, prompted by Alex Zhu (who also paid me for reviewing the work). The main CTMU paper is "The Cognitive-Theoretic Model of the Universe: A New Kind of Reality Theory".

CTMU has a high-IQ mystique about it: if you don't get it, maybe it's because your IQ is too low. The paper itself is dense with insights, especially the first part. It uses quite a lot of nonstandard terminology (partially because the author is outside the normal academic system), having few citations [...]

---

First published:

Source:

Linkpost URL:
https://unstablerontology.substack.com/p/the-cognitive-theoretic-model-of

Narrated by TYPE III AUDIO.

He was 90 years old.

His death was confirmed by his stepdaughter Deborah Treisman, the fiction editor for the New Yorker. She did not say where or how he died.

---

First published:

Source:

Linkpost URL:
https://www.washingtonpost.com/obituaries/2024/03/27/daniel-kahneman-dead/

Narrated by TYPE III AUDIO.