Epistemic status: these are my own opinions on AI risk communication, based primarily on my own instincts on the subject and discussions with people less involved with rationality than myself. Communication is highly subjective and I have not rigorously A/B tested messaging. I am even less confident in the quality of my responses than in the correctness of my critique.

If they turn out to be true, these thoughts can probably be applied to all sorts of communication beyond AI risk.

Lots of work has gone into trying to explain AI risk to laypersons. Overall, I think it's been great, but there's a particular trap that I've seen people fall into a few times. I'd summarize it as simplifying and shortening the text of an argument without enough thought for the information content. It comes in three forms. One is forgetting to adapt concepts for someone with a far [...]

---

Outline:

(01:14) Failure to Adapt Concepts

(03:41) Failure to Filter Information

(05:09) Failure to Sound Like a Human Being

(07:23) Summary

The original text contained 4 images which were described by AI.

---

First published:

September 10th, 2024

Source:

https://www.lesswrong.com/posts/CZQYP7BBY4r9bdxtY/the-best-lay-argument-is-not-a-simple-english-yud-essay

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In a recent appearance on Conversations with Tyler, famed political forecaster Nate Silver expressed skepticism about AIs replacing human forecasters in the near future. When asked how long it might take for AIs to reach superhuman forecasting abilities, Silver replied: “15 or 20 [years].”

In light of this, we are excited to announce “FiveThirtyNine,” an AI forecasting bot. Our bot, built on GPT-4o, provides probabilities for any user-entered query, including “Will Trump win the 2024 presidential election?” and “Will China invade Taiwan by 2030?” Our bot performs better than experienced human forecasters and performs roughly the same as (and sometimes even better than) crowds of experienced forecasters; since crowds are for the most part superhuman, FiveThirtyNine is in a similar sense. (We discuss limitations later in this post.)

Our bot and other forecasting bots can be used in a wide variety of contexts. For example, these AIs could help [...]

---

Outline:

(01:33) Quick Links

(01:43) Problem

(02:24) FiveThirtyNine

(07:56) Vision

(10:30) Conclusion

The original text contained 7 images which were described by AI.

---

First published:

September 9th, 2024

Source:

https://www.lesswrong.com/posts/4kuXNhPf9FBwok7tK/ai-forecasting-bots-incoming

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Midjourney image

For this week's post, I thought I’d mess around a bit with the CellXGene tool provided by the Chan Zuckerberg Institute.

It's based on a big dataset of individual cells, classified by tissue, cell type, and disease state, and their gene expression profiles (single-cell RNA counts).

You can automatically compare how gene expression looks different between sick and healthy individuals, for a variety of diseases, and drill down into which cells/tissues are different and how.

It's a fascinating toy and a great way to generate hypotheses.

Here, I’ll do it for Alzheimer's, comparing 138,438 Alzheimer's brain cells to 9,203,998 normal/healthy brain cells to see what the most “differentially expressed” genes are, and what that might tell us about how the disease works.

Top Hits

LINC01609

1.6x overexpressed in Alzheimer's, d =4.203

This is a non-protein coding RNA. Typically most expressed in the testis. In [...]

---

Outline:

(01:13) Top Hits

(01:16) LINC01609

(01:21) 1.6x overexpressed in Alzheimer's, d =4.203

(01:50) SLC26A3

(01:55) 10.6x overexpressed in Alzheimer's, d = 3.310

(04:06) RASGEF1B

(04:10) 5.5x overexpressed in Alzheimer's, d=3.267

(05:17) LINGO1

(05:20) 3.9x overexpressed in Alzheimer's, d=2.799

(06:11) INO80D

(06:15) 2x overexpressed in Alzheimer's, d =2.244

(07:21) What's Going On Here?

(09:55) What Does CellxGene Get You?

The original text contained 2 images which were described by AI.

---

First published:

September 6th, 2024

Source:

https://www.lesswrong.com/posts/3jXqvHej48Q3t8yjv/fun-with-cellxgene

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Pew

recently commissioned Gallup to run a poll in Israel on attitudes

to social media censorship. They found high support for banning

various kinds of speech:

The fourth question, especially, is disturbing: 59% want to ban

publicly expressing sympathy for civilians in Gaza? Since the

polling was conducted in Hebrew and Arabic, however, this got me wondering

whether translation might be contributing. I went to check, and Pew

did publish

the questions:

Q123c. Thinking about the war between Israel and Hamas, do

you think people should or should not be allowed to post each of

the following on social media sites? c. Posts that express

sympathy for civilians in Gaza

A friend of a friend guessed

they might have used סימפטיה

("simpatia"):

I don't *know* because they're not transparent, but I [...]

The original text contained 1 image which was described by AI.

---

First published:

September 8th, 2024

Source:

https://www.lesswrong.com/posts/28eyDuZ6FKrHpQ2FK/pollsters-should-publish-question-translations

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Written by Spencer Greenberg & Amber Dawn Ace for Asimov Press.

In 1994, the U.S. Congress passed the largest crime bill in U.S. history, called the Violent Crime Control and Law Enforcement Act. The bill allocated billions of dollars to build more prisons and hire 100,000 new police officers, among other things. In the years following the bill's passage, violent crime rates in the U.S. dropped drastically, from around 750 offenses per 100,000 people in 1990 to under 400 in 2018.

A chart showing U.S. crime rates over time. The data and annotation are real, but the implied story is not. Credit: Authors.

But can we infer, as this chart seems to ask us to, that the bill caused the drop in crime?

As it turns out, this chart wasn’t put together by sociologists or political scientists who’ve studied violent crime. Rather, we—a mathematician and a writer—devised it to make [...]

---

Outline:

(02:51) Misleading Charts

(13:55) Manipulating Reality

The original text contained 3 images which were described by AI.

---

First published:

September 8th, 2024

Source:

https://www.lesswrong.com/posts/dwvusp6wr9GfZYkrK/on-fables-and-nuanced-charts

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In my last post, I wrote that no resource out there exactly captured my model of epistemology, which is why I wanted to share a half-baked version of it.

But I do have one book which I always recommend to people who want to learn more about epistemology: Inventing Temperature by Hasok Chang.

To be very clear, my recommendation is not just to get the good ideas from this book (of which there are many) from a book review or summary — it's to actually read the book, the old-school way, one word at a time.

Why? Because this book teaches you the right feel, the right vibe for thinking about epistemology. It punctures the bubble of sterile non-sense that so easily pass for “how science works” in most people's education, such as the “scientific method”. And it does so by demonstrating how one actually makes progress in epistemology [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

September 8th, 2024

Source:

https://www.lesswrong.com/posts/TbaCa7sY3GxHBcXTd/my-number-1-epistemology-book-recommendation-inventing

---

Narrated by TYPE III AUDIO.

Our new video is an adaptation of That Alien Message, by @Eliezer Yudkowsky. This time, the text has been significantly adapted, so I include it below.

Part 1

Picture a world just like ours, except the people are a fair bit smarter: in this world, Einstein isn’t one in a million, he's one in a thousand. In fact, here he is now. He's made all the same discoveries, but they’re not quite as unusual: there have been lots of other discoveries. Anyway, he's out one night with a friend looking up at the stars when something odd happens. [visual: stars get brighter and dimmer, one per second. The two people on the hill look at each other, confused]

The stars are flickering. And it's just not a hallucination. Everyone's seeing it.

And so everyone immediately freaks out and panics! Ah, just kidding, the people of this world are [...]

---

Outline:

(00:16) Part 1

(06:22) Part 2

(09:53) Part 3

(11:58) Part 4

---

First published:

September 7th, 2024

Source:

https://www.lesswrong.com/posts/Q9omyL3qooXdjnyZn/that-alien-message-the-animation

---

Narrated by TYPE III AUDIO.

“A Reader's Manifesto” is a July 2001 Atlantic piece by B.R. Myers that I've returned to many times. He complains about the inaccessible pretension of the highbrow literary fiction of his day. The article is mostly a long list of critiques of various quotes/passages from well-reviewed books by famous authors. It's hard to accuse him of cherry-picking since he only targets passages that reviewers singled out as unusually good.

Some of his complaints are dumb but the general idea is useful: authors try to be “literary” by (1) avoiding a tightly-paced plot that could evoke “genre fiction” and (2) trying to shoot for individual standout sentences that reviewers can praise, using a shotgun approach where many of the sentences are banal or just don’t make sense.

Here are some excerpts of his complaints. Bolding is always mine.

The “Writerly” Style

He complains that critics now dismiss too much good [...]

---

Outline:

(00:57) The “Writerly” Style

(03:53) 4 Types of Bad Prose

(04:10) “Evocative” Prose

(08:12) “Muscular” Prose

(11:15) “Edgy” Prose

(19:11) “Spare” Prose

(21:37) Some Prose He Thinks Is Good

---

First published:

September 6th, 2024

Source:

https://www.lesswrong.com/posts/NiTuqw6LRFifqzRRH/excerpts-from-a-reader-s-manifesto

---

Narrated by TYPE III AUDIO.

Personally, I suspect the alignment problem is hard. But even if it turns out to be easy, survival may still require getting at least the absolute basics right; currently, I think we're mostly failing even at that.

Early discussion of AI risk often focused on debating the viability of various elaborate safety schemes humanity might someday devise—designing AI systems to be more like “tools” than “agents,” for example, or as purely question-answering oracles locked within some kryptonite-style box. These debates feel a bit quaint now, as AI companies race to release agentic models they barely understand directly onto the internet.

But a far more basic failure, from my perspective, is that at present nearly all AI company staff—including those tasked with deciding whether new models are safe to build and release—are paid substantially in equity, the value of which seems likely to decline if their employers stop building and [...]

---

First published:

September 7th, 2024

Source:

https://www.lesswrong.com/posts/sMBjsfNdezWFy6Dz5/pay-risk-evaluators-in-cash-not-equity

---

Narrated by TYPE III AUDIO.

Diego Caples ([email protected])

Rob Neuhaus ([email protected])

Introduction

In principle, neuron activations in a transformer-based language model residual stream should be about the same scale. In practice, however the dimensions unexpectedly widely vary in scale. Mathematical theories of the transformer architecture do not predict this. They expect rotational equivariance within a model, where one dimension is no more important than any other. Is there something wrong with our reasonably informed intuitions of how transformers work? What explains these outlier channels?

Previously, Anthropic researched the existence of these privileged basis dimensions (dimensions more important / larger than expected) and ruled out several causes. By elimination, they reached the hypothesis that per-channel normalization in the Adam optimizer was the cause of privileged basis. However, they did not prove this was the case.

We conclusively show that Adam causes outlier channels / privileged basis within the transformer residual stream. When replacing the Adam [...]

---

Outline:

(00:17) Introduction

(02:06) Background

(02:09) Recommended Reading

(02:20) More About Anthropic's Work

(02:54) Adam vs SGD, and Rotational Equivariance

(03:52) Kurtosis

(04:43) TinyStories

(05:14) Experiments

(05:17) Replicating Outlier Channels at Small Scale

(06:03) Training an LM with SGD

(07:09) Conclusions

(08:37) Future Research

The original text contained 2 images which were described by AI.

---

First published:

September 6th, 2024

Source:

https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optimizer-causes-privileged-basis-in-transformer

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.