Inspired by a house party inspired by Scott Alexander.

By the time you arrive in Berkeley, the party is already in full swing. You’ve come late because your reading of the polycule graph indicated that the first half would be inauspicious. But now you’ve finally made it to the social event of the season: the Every Bay Area House Party-themed house party.

The first order of the evening is to get a color-coded flirting wristband, so that you don’t incur any accidental micromarriages. You scan the menu of options near the door. There's the wristband for people who aren’t interested in flirting; the wristband for those want to be flirted with, but will never flirt back; the wristband for those who only want to flirt with people who have different-colored wristbands; and of course the one for people who want to glomarize disclosure of their flirting preferences. Finally you [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Epistemic status: the stories here are all as true as possible from memory, but my memory is so so.

This is going to be big

It's late Summer 2017. I am on a walk in the Mendip Hills. It's warm and sunny and the air feels fresh. With me are around 20 other people from the Effective Altruism London community. We’ve travelled west for a retreat to discuss how to help others more effectively with our donations and careers. As we cross cow field after cow field, I get talking to one of the people from the group I don’t know yet. He seems smart, and cheerful. He tells me that he is an AI researcher at Google DeepMind. He explains how he is thinking about how to make sure that any powerful AI system actually does what we want it [...]

---

Outline:

(00:18) This is going to be big

(01:23) This is going to be bad

(02:45) It's a long way off though

(03:52) This is fine

(05:11) It's probably something else in your life

(06:17) No-one in my org puts money in their pension

(07:17) Doom-vibes

(08:47) Maths might help

(10:30) A problem shared is…

(12:37) Hope

---

First published:

Source:

Linkpost URL:
https://seekingtobejolly.substack.com/p/no-one-in-my-org-puts-money-in-their

Narrated by TYPE III AUDIO.

Produced as part of the ML Alignment Theory Scholars Program - Winter 2023-24 Cohort under advisement of Lee Sharkey.

_DeclareMathOperator*{argmax}{arg,max} DeclareMathOperator*{argmin}{arg,min}_Sparse autoencoders are a method of resolving superposition by recovering linearly encoded “features” inside activations. Unfortunately, despite the great recent success of SAEs at extracting human interpretable features, they fail to perfectly reconstruct the activations. For instance, Cunningham et al. (2023) note that replacing the residual stream of layer 2 of Pythia-70m with the reconstructed output of an SAE increased the perplexity of the model on the Pile from 25 to 40. It is important for interpretability that the features we extract accurately represent what the model is doing.

In this post, I show how and why SAEs have a reconstruction gap due to ‘feature suppression’. Then, I look at a few ways to fix this while maintaining SAEs interpretability. By modifying and fine-tuning a pre-trained SAE, we [...]

---

Outline:

(01:26) Feature Suppression

(02:08) An illustrative example of feature suppression

(03:24) Feature suppression is a significant problem in current SAEs

(04:47) How can we fix feature suppression in trained SAEs?

(06:20) Fine-tuning Reduces Feature Suppression

(09:28) Activation Strength Causes Feature Suppression

(10:35) A theoretical example predicts frequency isnt a factor

(11:25) Fine-tuning does not fix regression dilution

(12:37) Experimental measurements agree for activation strength, disagree for frequency

(14:35) Conclusion

(15:31) Appendix

(15:34) Related Work

(16:29) Extended Data

(17:19) Summary Statistics for SAEs

(17:47) Low strength/high frequency features also rotate more

---

First published:

Source:

Narrated by TYPE III AUDIO.

TLDR: If you're a professor, PhD student, or industry researcher or engineer interested in starting research on AI safety, please request a call with Arkose! We are currently focused on and most helpful to mid-career people with machine learning expertise, who are relatively new to the advanced AI safety space.

The following is largely a linkpost for arkose.org, with a FAQ in the comments.

Arkose is an early-stage, fieldbuilding nonprofit with the mission of improving the safety of advanced AI systems. We focus on supporting researchers, engineers, and other professionals interested in contributing to AI safety, primarily by hosting personalized calls and public-facing resources.

In our 30-45 minute calls, we offer personalized advice to fit your level of interest and situation, whether you're an industry researcher or engineer, professor, or PhD student.

After discussing your specific needs and questions, we can:

• Help you explore funding and [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Cross-posted with light edits from Otherwise.

I think of us in some kind of twilight world as transformative AI looks more likely: things are about to change, and I don’t know if it's about to get a lot darker or a lot brighter.

Increasingly this makes me wonder how I should be raising my kids differently.

What might the world look like

Most of my imaginings about my children's lives have them in pretty normal futures, where they go to college and have jobs and do normal human stuff, but with better phones.

It's hard for me to imagine the other versions:

• A lot of us are killed or incapacitated by AI
• More war, pandemics, and general chaos
• Post-scarcity utopia, possibly with people living as uploads 
• Some other weird outcome I haven’t imagined

Even in the world where change is slower, more like the speed [...]

---

Outline:

(00:22) What might the world look like

(01:17) When?

(01:48) Is it even ok to have kids?

(04:02) In the meantime

(04:32) What skills or mindsets will be helpful?

(05:50) How much contact with AI?

(06:37) What we say to them

(07:11) CS Lewis in 1948

(07:46) Related writing

---

First published:

Source:

Narrated by TYPE III AUDIO.

New York Times (unpaywalled):

When the cryptocurrency exchange FTX declared bankruptcy about 15 months ago, it seemed few customers would recover much money or crypto from the platform. As John Ray III, who took over as chief executive during the bankruptcy, put it, “At the end of the day, we’re not going to be able to recover all the losses here.” He was countering Sam Bankman-Fried's repeated claims that he could get every customer their money back.

Well, it turns out, FTX lawyers told a bankruptcy judge this week that they expected to pay creditors in full, though they said it was not a guarantee and had not yet revealed their strategy.

The surprise turn of events is raising serious questions about what happens next. Among them: What does this mean for the lawsuits FTX has filed in an attempt to claw back billions [...]

---

First published:

Source:

Linkpost URL:
https://www.nytimes.com/2024/02/03/business/dealbook/what-happens-to-ftx-clawback-cases-if-the-company-repays-its-creditors.html?unlocked_article_code=1.VU0.QBLc.h36dKm_EgU8L&smid=url-share

Narrated by TYPE III AUDIO.

I'm trying to build my own art of rationality training, and I've started talking to various CFAR instructors about their experiences – things that might be important for me to know but which hadn't been written up nicely before.

This is a quick write up of a conversation with Andrew Critch about his takeaways. (I took rough notes, and then roughly cleaned them up for this. I don't know

"What surprised you most during your time at CFAR?

Surprise 1: People are profoundly non-numerate.

And, people who are not profoundly non-numerate still fail to connect numbers to life.

I'm still trying to find a way to teach people to apply numbers for their life. For example: "This thing is annoying you. How many minutes is it annoying you today? how many days will it annoy you?". I compulsively do this. There aren't things lying around in [...]

---

Outline:

(00:27) What surprised you most during your time at CFAR?

(03:36) What do other people seem surprised or confused about, which are important if theyre gonna do something rationality-training-ish

(06:20) You once told me that there were ~20 things a person needed to be generally competent. What were they?

---

First published:

Source:

Narrated by TYPE III AUDIO.

A sequel to qntm's Lena. Reading Lena first is helpful but not necessary.

We’re excited to announce the fourth annual MMindscaping competition! Over the last few years, interest in the art of mindscaping has continued to grow rapidly. We expect this year's competition to be our biggest yet, and we’ve expanded the prize pool to match. The theme for the competition is “Weird and Wonderful”—we want your wackiest ideas and most off-the-wall creations!

Competition rules

As in previous competitions, the starting point is a base MMAcevedo mind upload. All entries must consist of a single modified version of MMAcevedo, along with a written or recorded description of the sequence of transformations or edits which produced it. For more guidance on which mind-editing techniques can be used, see the Technique section below.

Your entry must have been created in the last 12 months, and cannot [...]

---

Outline:

(00:34) Competition rules

(01:40) Award criteria

(05:02) Prohibited submissions

(06:16) Prizes

---

First published:

Source:

Linkpost URL:
https://www.narrativeark.xyz/p/masterpiece

Narrated by TYPE III AUDIO.

Why aren’t there dojos that teach rationality?

―The Martial Art of Rationality by Eliezer Yudkowsky

For the last 6 months, I've been running a dojo that teaches rationality.

Why

I was at an ACX meetup and met an acolyte who grew up in an evangelical Christian community. He had recently discovered the Sequences and was really excited about this whole Rationality thing. He was very confident in Yudkowsky's teachings.

I asked him a couple questions and he realized his beliefs were full of holes. He wondered how he could understood so little. After all, he had read all of Yudkowsky's Sequences.

"I have read 100 books about chess," I said, "Surely I must be a grandmaster by now."

At that moment, he was enlightened.

The problem

The objective of rationality is to become right instead of wrong. Being wrong feels exactly like being right. We are not aware of [...]

---

Outline:

(00:20) Why

(00:57) The problem

(01:36) The solution

(02:51) The right conditions

(04:02) The right topics

(04:44) Want to try it out?

(05:26) What to expect if you participate

---

First published:

Source:

Narrated by TYPE III AUDIO.

I am seeking crowdsourced wisdom.

Suppose I[1] want to influence public opinion on a complicated, nuanced topic.[2]

And further suppose that one of the ways I want to do that is by participating actively in public discourse: posting short-form content, reacting to other people's short-form content, signal boosting good takes, thoughtfully rebutting bad takes, etc.

Even further suppose that the people I most want to reach are those who make and influence policy in the most technologically advanced countries in the world, the general voting public (because they vote for legislators), and the intelligentsia in fields related to my topic (because some of them advise policymakers).

On what platform(s)/in what outlet(s) should I be doing this in 2024?

And more specifically, does Twitter/X still matter, and how much? I am aware that many people have moved to mastodon or bluesky or whatever. Is there critical mass anywhere?

1. The original text contained 2 footnotes which were omitted from this narration.

   ---

First published:

Source:

Narrated by TYPE III AUDIO.