At least, if you happen to be near me in brain space.

What advice would you give your younger self?

That was the prompt for a class I taught at PAIR 2024. About a quarter of participants ranked it in their top 3 of courses at the camp and half of them had it listed as their favorite.

I hadn’t expected that.

I thought my life advice was pretty idiosyncratic. I never heard of anyone living their life like I have. I never encountered this method in all the self-help blogs or feel-better books I consumed back when I needed them.

But if some people found it helpful, then I should probably write it all down.

Why Listen to Me Though?

I think it's generally worth prioritizing the advice of people who have actually achieved the things you care about in life. I can’t tell you if that's me [...]

---

Outline:

(00:46) Why Listen to Me Though?

(04:22) Pick a direction instead of a goal

(12:00) Do what you love but always tie it back

(17:09) When all else fails, apply random search

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 38 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

This post has some ablation results around the thesis of the ICML 2024 Mech. Interp. workshop 1st prize winning paper: The Geometry of Categorical and Hierarchical Concepts in Large Language Models The main takeaway is that the orthogonality they observe in categorical and hierarchical concepts occurs practically everywhere, even at places where it really should not.

Overview of the original paper

A lot of the intuition and math around why they do what they do is shared in their previous work called The Linear Representation Hypothesis and the Geometry of Large Language Models, but let's quickly go over what the paper's core idea is:

They split the computation of a large language model (LLM) as:

_P(y mid [...]

---

Outline:

(02:42) Ablations

(03:37) Hierarchical features are orthogonal - but so are semantic opposites!?

(04:51) Categorical features form simplices - but so do totally random ones!?

(05:27) Orthogonality being ubiquitous in high dimensions

(06:39) Discussion and Future Work

The original text contained 4 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For a long time, when I heard "slow takeoff", I assumed it meant "takeoff that takes longer calendar time than fast takeoff." (i.e. what is now referred to more often as "short timelines" vs "long timelines."). I think Paul Christiano popularized the term, and it so happened he both expected to see longer timelines and smoother/continuous takeoff.

I think it's at least somewhat confusing to use the term "slow" to mean "smooth/continuous", because that's not what "slow" particularly means most of the time.

I think it's even more actively confusing because "smooth/continuous" takeoff not only could be faster in calendar time, but, I'd weakly expect this on average, since smooth takeoff means that AI resources at a given time are feeding into more AI resources, whereas sharp/discontinuous takeoff would tend to mean "AI tech doesn't get seriously applied towards AI development until towards the end."

I don't think this [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Part 0: "Previously, on Petrov Day..."

By Raymond Arnold

One year ago, many people on LessWrong received a DM asking them to choose the most important virtue of Petrov Day, with four listed options that we'd seen people argue for in previous years.

1. "Avoiding actions that noticeably increase chance that civilization is destroyed"
2. "Accurately reporting your epistemic state"
3. "Quickly orienting to novel situations"
4. "Resisting social pressure."

Then, people who chose one of those options were sent another message saying "Your virtue was in the minority, but you have the power to unilaterally choose the virtue of next year's Petrov Day by clicking a link."

The first person to click the Unilateral Virtue Link was a proponent of "Avoiding actions that noticeably increase the chance that the world will end." But, this virtue was actually in the majority[1]. The first unilateralist of a Virtue Minority was a proponent [...]

---

Outline:

(00:05) Part 0: Previously, on Petrov Day...

(00:12) By Raymond Arnold

(03:01) Part I: The 2024 Petrov Day Scenario

(03:07) By Ben Pace

(09:50) Part II: Stats and Feedback Form

(12:18) Part IV: Reflections

(12:23) Reflections by Raymond Arnold

(16:59) Reflections by Ben Pace

The original text contained 6 footnotes which were omitted from this narration.

The original text contained 14 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This graph is the biggest update to the AI alignment discourse since GPT-3

For those of you unfamiliar with the lore, prior to GPT-3, the feeling was that AGI would rapidly foom based on recursive-self improvement.

After GPT-3, it became clear that the first AGI would in reality be built in a large lab using a multi-billion dollar supercomputer and any idea that it simply "copy itself to the internet" is nonsense.

Under the GPT-3 regime, however, it was still plausible to assume that the first AGI would be able to simulate millions of human beings. This is because the training cost for models like GPT-3/4 is much higher than the inference cost.

However, COT/o1 reveals this is not true. Because we can scale both training and inference, the first AGI will not only cost billions of dollars to train, it will also cost millions of dollars to run [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously: The Fundamentals, The Gamblers, The Business

We have now arrived at the topics most central to this book, aka ‘The Future.’

Rationalism and Effective Altruism (EA)

The Manifest conference was also one of the last reporting trips that I made for this book. And it confirmed for me that the River is real—not just some literary device I invented. (6706)

Yep. The River is real.

I consider myself, among many things, a straight up rationalist.

I do not consider myself an EA, and never have.

This completes the four quadrants of the two-by-two of [does Nate knows it well, does Zvi knows it well]. The first two, where Nate was in his element, went very well. The third clearly was less exacting, as one would expect, but pretty good.

Now I have the information advantage, even more than I did [...]

---

Outline:

(00:16) Rationalism and Effective Altruism (EA)

(06:01) Cost-Benefit Analysis

(09:04) How About Trying At All

(10:11) The Virtues of Rationality

(11:56) Effective Altruism and Rationality, Very Different of Course

(24:37) The Story of OpenAI

(30:19) Altman, OpenAI and AI Existential Risk

(38:26) Tonight at 11: Doom

(01:00:39) AI Existential Risk: They’re For It

(01:07:42) To Pause or Not to Pause

(01:11:11) You Need Better Decision Theory

(01:15:27) Understanding the AI

(01:19:43) Aligning the AI

(01:23:50) A Glimpse of Our Possible Future

(01:28:16) The Closing Motto

---

First published:

Source:

Narrated by TYPE III AUDIO.

We're excited to announce the inaugural Australian AI Safety Forum, taking place on November 7-8, 2024, in Sydney, Australia. This event aims to foster the growth of the AI safety community within Australia.

Apply now! 

Key Details

• Dates: November 7-8, 2024
• Location: Sydney Knowledge Hub, The University of Sydney
• Website: https://aisafetyforum.au

Description

The Australian AI Safety Forum is a two-day interdisciplinary event scheduled for November 2024. This forum, the first of its kind in Australia, aims to discuss perspectives on technical AI safety and governance, and explore Australia's unique role in the global AI safety landscape. The event will be anchored around the International Scientific Report on the Safety of Advanced AI, highlighting its key content and examining its implications for Australia.

The forum is designed to encourage conversation and education between researchers, policy makers, and industry leaders. As such, we aim to catalyse an Australian community working [...]

---

Outline:

(00:21) Apply now!

(00:25) Key Details

(00:40) Description

(01:34) Who Should Apply

(02:28) Format

(02:56) Organising Committee

(03:22) Partners

(03:41) Application Process

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previously: The Fundamentals, The Gamblers

Having previously handled the literal gamblers, we are ready to move on to those who Do Business using Riverian principles.

Or at least while claiming to use Riverian principles, since Silicon Valley doesn’t fit into the schema as cleanly as many other groups. That's where we begin this section, starting at the highest possible conceptual level.

Time to talk real money.

Why Can You Do This Trade?

First law of trading: For you to buy, someone must sell. Or for you to sell, someone must buy. And there can’t be someone else doing the trade before you did it.

Why did they do that, and why did no one else take the trade first? Until you understand why you are able to do this trade, you should be highly suspicious.

“Every single thing we do, I can [...]

---

Outline:

(00:41) Why Can You Do This Trade?

(03:08) In a World of Venture Capital

(10:54) Short Termism Hypothesis

(12:42) Non-Determinism and its Discontents

(14:57) The Founder, the Fox and the Hedgehog

(17:11) The Team to Beat

(24:22) Silicon Valley Versus Risk

(35:14) The Keynesian Beauty Contest

(40:57) The Secret of Their Success is Deal Flow

(50:00) The Valley Beside the River

(53:07) Checkpoint Three

(53:37) Fun With SBF and Crypto Fraud

(01:01:53) Other Crypto Thoughts Unrelated to SBF

(01:04:50) Checkpoint Four

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Quarterly Performance Review, Autumn 1983

Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe his nerves. He only had one employee review left to get through, but he’d saved the hardest one for last.

He wasn’t upset about having to dismiss Lieutenant Colonel Petrov—he couldn’t wait to be rid of the little shit—but he couldn’t shake the feeling that something was amiss. He took a swig from his flask.

“Stanislav, you can come in now,” Yuri shouted as he opened the door and nearly smashed Stanislav Petrov in the face. “Have a seat,” he said.

“Yes, sir. Thank you sir. Overjoyed to be here as always,” Stanislav said.

“The purpose of this meeting is for us to discuss various concerns that have emerged about your performance over the past several months,” said Yuri. “Looking through your chart, what I’m seeing [...]

---

Outline:

(00:04) Quarterly Performance Review, Autumn 1983

(07:27) Quarterly Performance Review, Winter 1983

---

First published:

Source:

Narrated by TYPE III AUDIO.

This research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.

This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated since then.

We study the effects of perturbing Transformer activations, building upon recent work by Gurnee, Lindsey, and Heimersheim & [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.