Epistemic Status: Soldier mindset. These are not our actual positions, these are positions we were randomly assigned by a coin toss, and for which we (quickly) searched for the strongest arguments we could find over the course of ~1hr 45mins. That said, this debate got a little messy between our performed positions and our personal ones.

Sides: Ben is arguing against getting a college degree, and Saul is arguing for. (This is a decision Saul is currently making for himself!)

Reading Order: Ben and Saul drafted each round of statements simultaneously. This means that each of Ben's statements you read are written without having read Saul's statements that are immediately proceeding. (This does not apply to the back-and-forth interview.)

Saul's Opening Statement

first — i do think there's a qualitative difference between the position “getting an undergrad degree is good” vs “getting the typical undergrad experience is good.” [...]

---

Outline:

(00:59) Sauls Opening Statement

(04:31) Bens Opening Statement

(11:31) Ben interviews Saul

(17:41) Saul interviews Ben

(24:33) Sauls Rebuttal

(26:56) Bens Rebuttal

(30:22) Sauls Closing Statement

(34:26) Bens Closing Statement

---

First published:

Source:

Narrated by TYPE III AUDIO.

Toby Ord recently published a nice piece On the Value of Advancing Progress about mathematical projections of far-future outcomes given different rates of progress and risk levels. The problem with that and many arguments for caution is that people usually barely care about possibilities even twenty years out.

We could talk about sharp discounting curves in decision-making studies, and how that makes sense given evolutionary pressures in tribal environments. But I think this is pretty obvious from talking to people and watching our political and economic practices.

Utilitarianism is a nicely self-consistent value system. Utilitarianism pretty clearly implies longtermism. Most people don't care that much about logical consistency,[1] so they are happily non-utilitarian and non-longtermist in a variety of ways. Many arguments for AGI safety are longtermist, or at least long-term, so they're not going to work well for most of humanity.

This is another fairly obvious [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 33 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

A Defense of Open-Minded Updatelessness, with applications to alignment.

This work owes a great debt to many conversations with Sahil, Martín Soto, and Scott Garrabrant.

Sahil and I will have a public discussion about this this Wednesday (August 14) at 11am EDT; join via this link.

You can support my work on Patreon.

Iterated Counterfactual Mugging On a Single Coinflip

Iterated counterfactual mugging on a single coinflip begins like a classic counterfactual mugging, with Omega approaching you, explaining the situation, and asking for your money. Let's say you buy the classic UDT idea, so you happily give Omega your money.

Next week, Omega appears again [...]

---

Outline:

(00:44) Iterated Counterfactual Mugging On a Single Coinflip

(03:28) UDT vs Learning

(09:24) Possible Critiques

(19:29) Bargaining

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Alex Lawsen used a great metaphor on the 80k After Hours podcast:[1]

[1:38:14] …you’re rowing a boat on your own, and you’re trying to get somewhere, and you’ve got some map that you need to look at to see where you’re going, I imagine like a map and compass. […] When you’re rowing, you’re facing back; you can’t see where you’re going. You’ve just got to sit there and pull both of the oars, and do that a bunch of times, and then the boat goes forwards. […] You steer [… by pulling] harder with one side, something like that.

I can imagine […] you sitting forwards in the boat, and trying to hold the map with your left hand while it's gripping one oar, and trying to hold the compass with your right hand while it's gripping the other; pushing them rather than [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Transcranial focused ultrasound neuromodulation — altering the brain's activity with low-intensity ultrasound — is really exciting .

It allows us to manipulate arbitrary regions of the brain without surgery, potentially replacing the (brain-damaging) electrode implants currently used for serious neurological conditions like epilepsy and Parkinson's, and potentially also expanding applications of brain stimulation to milder conditions not worth the risks of brain surgery, like mental illness, addiction, or chronic pain.

The field is rapidly growing, and since I wrote my earlier post series there have been quite a few human studies published. Here's a systematic overview of all the human studies published in 2024, by target brain region.

Headline Results

This year's papers further confirm, to start with, that ultrasound does things to brain activity, if that was still in doubt, and that it is safe enough [...]

---

Outline:

(00:59) Headline Results

(03:04) Amygdala

(04:20) Motor Cortex

(05:51) Nucleus Accumbens

(06:20) Posterior Cingulate Cortex

(07:38) Posterior Insula

(08:02) Prefrontal Cortex

(08:29) Thalamus

(10:01) Ventral Capsule/Striatum

(10:23) Visual Cortex

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In September 2023, Max Tegmark and Steve Omohundro proposed "Provably Safe AI" as a strategy for AI Safety. In May 2024, a larger group delineated the broader concept of "Guaranteed Safe AI" which includes Provably Safe AI and other related strategies. In July, 2024, Ben Goldhaber and Steve discussed Provably Safe AI and its future possibilities, as summarized in this document.

Background

In June 2024, ex-OpenAI AI Safety Researcher Leopold Aschenbrenner wrote a 165-page document entitled "Situational Awareness, The Decade Ahead" summarizing AI timeline evidence and beliefs which are shared by many frontier AI researchers. He argued that human-level AI is likely by 2027 and will likely lead to superhuman AI in 2028 or 2029. "Transformative AI" was coined by Open Philanthropy to describe AI which can "precipitate a transition comparable to the agricultural or industrial revolution". There appears to be a significant probability that Transformative AI may be [...]

---

Outline:

(00:35) Background

(03:43) Whats next?

(07:32) Some Useful Projects

---

First published:

Source:

Narrated by TYPE III AUDIO.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I stopped following the news back when Trump (first?)[1] got elected. The amount of attention put on a foreign[2] election was staggering, with normal media saccades replaced by Monk-level single-mindedness. Trump was the permanent spotlight for months.

The media's fixation on Trump had interesting downstream effects. My peer groups — normally a dynamic bunch — turned into a bunch of snide gossipmongers. Every day was Trump-day, with shared outrage being the primary source of connection. People scored points by retelling outrageous news, parroting hot takes and sharing edgy memes.

Focusing on judgment and outrage was unhealthy for me. I got addicted to the drama, allowing outrage to outcompete healthy forms of relating. I felt disconnected from my friends, got irritated more often, and had an increase in pessimistic thought patterns.

Around this time, I had a coworker who was always grumpy — always complaining about [...]

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A few of my new-parent friends started having terrible gut problems and figured they have colon cancer. Their doctors agreed it's the first thing to check. But the colonoscopies were negative for colon cancer. The tissue was inflamed though. One doctor called this "pre-cancer" (??)

Hmm what could be causing inflammation in the colon, but wouldn't show up on camera after you fasted and had medically-induced diarrhea for 24 hours?

The babies were born over a year before symptoms appeared, so it can't be related to pregnancy. No change in diet. No family history.

What happens a year or two after a kid is born? They go outside and immediately eat as much dirt as they can. What lives in dirt? Everything!

Me and my boy's mother ate some combantrin 3 months ago and have been clear since.

I'm currently trying to convince my [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

At last. Highlights: some details on Preparedness Framework evals + evals by METR and Apollo.

Preparedness framework evaluations

You should follow the link and read this section.

Brief comments:

• Cyber: the setup sounds good (but maybe substantially more powerful scaffolding/prompting is possible). Separately, I wish OpenAI shared the tasks (or a small random sample of them) or at least said more about where they came from. (Recall that DeepMind shared CTF tasks.)
• Bio uplift: GPT-4o clearly boosts users on biological threat creation tasks — OpenAI doesn't say that but shows a graph. (It continues to be puzzling that novices score similarly to experts.) (I kinda worry that this is the wrong threat model—most bio risk from near-future models comes from a process that looks pretty different from a bigger boost to users like these—but I don't have better ideas for evals.)
• Persuasion: unclear whether substantially [...]

---

Outline:

(00:16) Preparedness framework evaluations

(01:34) Third party assessments

(03:26) Context

---

First published:

Source:

Narrated by TYPE III AUDIO.