There are so many important efforts to make the world better that are

significantly limited by funding, and it would be great if we could

have a culture where significant and thoughtful giving was normal and

common. It's hard to build that sort of norm if people keep their

giving private, however, and so I've

long been

an advocate of being public about your giving. I

list my donations (jointly

with

Julia) and have taken

Giving What We Can's

10% Pledge (also

jointly with Julia).

In July GWWC

suggested people put the "small orange diamond" symbol (🔸)

in their usernames on social media to show that they've pledged.

Here's how the EA Forum describes this on the profile editing page:

This digital symbol reminds me of the physical Symbolic

Beads of Raikoth. In an older Scott Alexander post [...]

The original text contained 1 image which was described by AI.

---

First published:

November 7th, 2024

Source:

https://www.lesswrong.com/posts/tLMrm7ELg2dPmP2a7/signaling-with-small-orange-diamonds

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is an interim report sharing preliminary results. We hope this update will be useful to related research occurring in parallel.

Executive Summary

• Problem: Qwen1.5 0.5B Chat SAEs trained on the pile (webtext) fail to find sparse, interpretable reconstructions of the refusal direction from Arditi et al. The most refusal-related latent we find is coarse grained and underperforms the refusal direction at steering tasks.
  • This is disappointing. The point of an SAE is to find meaningful concepts. If it can’t sparsely reconstruct the important refusal direction, then that means it's either missing the relevant concepts, or these are shattered across many latents.
• Solution: Training a new SAE on a chat-specific dataset, LmSys-Chat-1M, finds a significantly sparser, more faithful, and interpretable reconstruction of the “refusal direction”.
  • The LmSys SAE is also more capable of finding interpretable “refusal” latents that we can use to effectively steer the model to bypass refusals.
• We [...]

---

Outline:

(00:19) Executive Summary

(02:07) Introduction

(04:10) Methodology: Training chat-data specific SAEs

(07:42) Evaluating chat SAEs on the refusal direction reconstruction task

(08:37) Chat data SAEs find more faithful refusal direction reconstructions

(11:11) Chat data SAEs find sparser refusal direction reconstructions

(13:22) Chat data SAEs find more interpretable decompositions of the refusal direction

(16:09) Chat data SAE refusal latents are better for steering

(21:03) The chat dataset is more important than the chat model

(23:44) Related Work

(25:45) Conclusion

(26:32) Limitations

(27:24) Future work

(28:25) Citing this work

(28:36) Author Contributions Statement

(29:23) Acknowledgments

The original text contained 1 footnote which was omitted from this narration.

---

First published:

November 7th, 2024

Source:

https://www.lesswrong.com/posts/rtp6n7Z23uJpEH7od/saes-are-highly-dataset-dependent-a-case-study-on-the

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As Americans know, the electoral college gives disproportionate influence to swing states, which means a vote in the extremely blue state of California was basically wasted in the 2024 election, as are votes in extremely red states like Texas, Oklahoma, and Louisiana. State legislatures have the Constitutional power to assign their state's electoral votes. So why don't the four states sign a compact to assign all their electoral votes in 2028 and future presidential elections to the winner of the aggregate popular vote in those four states? Would this even be legal?

The population of CA is 39.0M (54 electoral votes), and the population of the three red states is 38.6M (55 electoral votes). The combined bloc would control a massive 109 electoral votes, and would have gone for Biden in 2020 and Trump in 2024. Every state has an incentive to sign to increase their voters' influence in [...]

---

First published:

November 6th, 2024

Source:

https://www.lesswrong.com/posts/ABYqFqd3wzgguexjT/should-ca-tx-ok-and-la-merge-into-a-giant-swing-state-just

---

Narrated by TYPE III AUDIO.

I think Instruction-following AGI is easier and more likely than value aligned AGI, and that this accounts for one major crux of disagreement on alignment difficulty. I got several responses to that piece that didn't dispute that intent alignment is easier, but argued we shouldn't give up on value alignment. I think that's right. Here's another way to frame the value of personal intent alignment: we can use a superintelligent instruction-following AGI to solve full value alignment.

This is different than automated alignment research; it's not hoping tool AI can help with our homework, it's making an AGI smarter than us in every way do our homework for us. It's a longer term plan. Having a superintelligent, largely autonomous entity that just really likes taking instructions from puny humans is counterintuitive, but it seems both logically consistent. And it seems technically achievable on the current trajectory - if we [...]

---

First published:

November 5th, 2024

Source:

https://www.lesswrong.com/posts/587AsXewhzcFBDesH/intent-alignment-as-a-stepping-stone-to-value-alignment

---

Narrated by TYPE III AUDIO.

Me: I think it's possible that we might not need to figure out biotechnological advances for how we can create superbabies through embryo selection or so, but that actually they might already be around 1000 or more superbabies born on earth per year, only they happen to be born into an environment where they don't get educated and don't learn the necessary skills to advance science. By superbabies I mean genetic potential for intelligence of >=+7std on the human distribution.

Imaginary person (IP): What, how could that be true? There being some hunter tribe of genetical supergeniuses which are almost not communicating with the rest of humanity?

Me: Sorta yeah.

IP: And where would those superbabies be?

Me: Here:

.

.

.

.

.

IP: Are you joking?

Me: No. See this excellent reddit post (or my wrapper post around it) for why it's IMO a pretty [...]

---

Outline:

(02:29) Can someone please look into this?

(03:15) Cautionary notes

(04:08) Sidenote on how orcas (and pilot whales) could be useful datapoints for AI alignment

The original text contained 6 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

November 5th, 2024

Source:

https://www.lesswrong.com/posts/vKM4CTjz5fPB7vznb/an-alternative-approach-to-superbabies

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(Btw everything I write here about orcas also applies to a slightly lesser extent to pilot whales (especially long finned ones)[1].)

(I'm very very far from an orca expert - basically everything I know about them I learned today.)

I always thought that bigger animals might have bigger brains than humans but not actually more neurons in their neocortex (like elephants) and that number of neurons in the neocortex or prefrontal cortex might be a good inter-species indicator of intelligence for mammalian brains.[2] Yesterday I discovered that orcas actually have 2.05 times as many neurons in their neocortex[3] than humans from this wikipedia list. Interestingly though, given my pretty bad model of how intelligent some species are, the "number of neurons in neocortex" still seems like a proxy that doesn't perform too badly on the wikipedia list.

Orca brains are not just larger but also more strongly folded.

Orcas [...]

The original text contained 6 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

November 4th, 2024

Source:

https://www.lesswrong.com/posts/gRShuaWgKjizM4xPM/could-orcas-be-trained-to-be-smarter-than-humans

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's a concept I think about when teaching, which I call Graceful Degradation. The basic idea is, how well does this lesson work if someone doesn’t remember it very well or if I teach it badly?

I.

Consider throwing a punch.

Make a fist with your thumb on the outside of your fingers, because you don’t want to pop your thumb. Place your feet shoulder width apart and then take a comfortable step forward with one foot and plant yourself solidly. Bring your fist close to your chest or face, then move it in a straight line from there to your opponent so you waste as little motion as possible. You want to make contact with the first knuckles of your pointer and middle fingers. Don’t hit someone in the face unless you’re wearing boxing gloves, particularly the jaw or mouth since that's one big crumple zone.

Different martial [...]

---

Outline:

(00:16) I.

(01:39) 2.

(03:50) 3.

---

First published:

November 5th, 2024

Source:

https://www.lesswrong.com/posts/N65uQ5RP6dYEC6CHW/graceful-degradation

---

Narrated by TYPE III AUDIO.

(Epistemic status: I spoke simply / without "appears to" hedges, but I'm not sure of this at all.)

I’m confused why we keep getting scissors statements as our Presidential candidates, but we do. (That is: the candidates seem themselves to break many minds/conversations.)

A toy model:[1]

Take two capacities, A and B. Ideally anti-correlated.

Craft two candidates:

• Candidate X, who seems acceptable if you’re A-blind (if you have a major gap in your situation awareness near A).
• Candidate Y, who seems acceptable if you’re B-blind (if you have a major gap in your situation awareness near B).

Now let voters talk.

“How can you possibly vote for X, given how it’ll make a disaster on axis A?”, asks Susan. (She is B-blind, which is part of why she is so confused/irate/loud here.) Susan inquires in detail. She (accurately) determines the staunchest X-voters don't understand A, and (understandably, but [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

November 6th, 2024

Source:

https://www.lesswrong.com/posts/FkYAYQQig4FDTN6r5/scissors-statements-for-president

---

Narrated by TYPE III AUDIO.

Hello! I'm looking for community members to read speeches at the Bay Winter Solstice event this year. If you're interested, please email me by the end of November 17 with the speeches you're potentially interested in. I'll ask you to record yourself reading it (no need to be particularly polished).

I am currently auditioning for:

• Thanksgiving Prayer
• I Have Seen The Tops Of Clouds*
• Self-Compassion*
• What Resembles The Grave But Isn't
• 500 Million But Not A Single One More
• RMS Carpathia
• The Gift We Give To Tomorrow*
• An original piece about the malaria vaccine
• An original piece about the abolition of slavery

(The list is tentative and does not include all speeches at Solstice.)

For starred items, I am only including an excerpt of the speech, so please email me to get the excerpt I'm using before you record!

As a reminder, the event itself will [...]

---

First published:

November 4th, 2024

Source:

https://www.lesswrong.com/posts/DoT6G6raWWuSqpdR7/bay-winter-solstice-2024-speech-auditions

---

Narrated by TYPE III AUDIO.

In our jobs as AI safety researchers, we think a lot about what it means to have reasonable beliefs and to make good decisions. This matters because we want to understand how powerful AI systems might behave. It also matters because we ourselves need to know how to make good decisions in light of tremendous uncertainty about how to shape the long-term future.

It seems to us that there is a pervasive feeling in this community that the way to decide which norms of rationality to follow is to pick the ones that win. When it comes to the choice between CDT vs. EDT vs. LDT…, we hear we can simply choose the one that gets the most utility. When we say that perhaps we ought to be imprecise Bayesians, and therefore be clueless about our effects on the long-term future, we hear that imprecise Bayesianism is “outperformed” by [...]

---

Outline:

(01:31) “Winning” gives little guidance

(01:50) Avoiding dominated strategies

(04:36) Heuristics

(11:27) Non-pragmatic principles

(15:25) Conclusion

(16:48) Acknowledgments

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

November 5th, 2024

Source:

https://www.lesswrong.com/posts/QxoGM89f8zr3JmNrz/winning-isn-t-enough

---

Narrated by TYPE III AUDIO.