Your mind is a prediction machine, constantly trying to anticipate the world around you and altering its forecasts based on new information. It's always doing this as a background process. But what would happen if you deliberately trained this skill? Could you get better at predicting your projects, your life, and the future?

Sadly, I don’t have a crystal ball, but I have found trying to see the future valuable. I can't be certain about what will happen, but by deliberately practising the skill of prediction, I can create feedback loops that allow me to better anticipate what the future will bring. This has indeed helped me become happier, better to interact with, and more impactful.

Concretely, making probabilistic predictions has helped me:

• Improve my productivity
• Make better decisions about my career and living situation
• Set and make progress towards my goals
• Prioritise work tasks
• Communicate project timelines [...]

---

Outline:

(01:34) Which questions are worth asking?

(10:53) How to track your predictions

The original text contained 20 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 149 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

ARC recently released our first empirical paper: Estimating the Probabilities of Rare Language Model Outputs. In this work, we construct a simple setting for low probability estimation — single-token argmax sampling in transformers — and use it to compare the performance of various estimation methods. ARC views low probability estimation as a potential technique for mitigating worst-case behavior from AI, including deceptive alignment; see our previous theory post on estimating tail risks in neural networks for further discussion.

Outline of this post:

---

Outline:

(01:32) Problem statement

(03:24) Estimation methods

(04:57) Importance sampling

(05:57) Independent Token Gradient Importance Sampling (ITGIS)

(07:16) Metropolis—Hastings Importance Sampling (MHIS)

(08:50) Activation extrapolation

(09:21) Quadratic Logit Decomposition (QLD)

(11:06) Gaussian Logit Difference

(11:41) Empirical results

(14:08) Discussion

(14:11) Motivation

(16:15) Importance sampling versus activation extrapolation

(17:15) Limitations

(18:54) Related work

(19:56) Future directions

The original text contained 7 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Of all the cognitive tools our ancestors left us, what's best? Society seems to think pretty highly of arithmetic. It's one of the first things we learn as children. So I think it's weird that only a tiny percentage of people seem to know how to actually use arithmetic. Or maybe even understand what arithmetic is for. Why?

I think the problem is the idea that arithmetic is about “calculating”. No! Arithmetic is a world-modeling technology. Arguably, it's the best world-modeling technology: It's simple, it's intuitive, and it applies to everything. It allows you to trespass into scientific domains where you don’t belong. It even has an amazing error-catching mechanism built in.

One hundred years ago, maybe it was important to learn long division. But the point of long division was to enable you to do world-modeling. Computers don’t make arithmetic obsolete. If anything, they do the opposite. Without [...]

---

Outline:

(01:17) Chimps

(06:18) Big blocks

(09:34) More big blocks

The original text contained 5 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains, and we avoided negative outcomes and solved the control and alignment problems. As he notes, a lot of pretty great things would then be super doable.

Anthropic also offers us improvements to its Responsible Scaling Policy (RSP, or what SB 1047 called an SSP). Still much left to do, but a clear step forward there.

Daniel Kokotajlo and Dean Ball have teamed up on an op-ed for Time on the need for greater regulatory transparency. It's very good.

Also, it's worth checking out the Truth Terminal saga. It's not as scary as it might look at first glance, but it is definitely [...]

---

Outline:

(01:01) Language Models Offer Mundane Utility

(05:10) Language Models Don’t Offer Mundane Utility

(11:21) Deepfaketown and Botpocalypse Soon

(19:52) They Took Our Jobs

(20:33) Get Involved

(20:48) Introducing

(21:58) In Other AI News

(26:08) Truth Terminal High Weirdness

(34:54) Quiet Speculations

(44:45) Copyright Confrontation

(45:02) AI and the 2024 Presidential Election

(46:02) The Quest for Sane Regulations

(51:00) The Week in Audio

(53:40) Just Think of the Potential

(01:15:09) Reactions to Machines of Loving Grace

(01:25:32) Assuming the Can Opener

(01:32:32) Rhetorical Innovation

(01:35:41) Anthropic Updates its Responsible Scaling Policy (RSP/SSP)

(01:41:35) Aligning a Smarter Than Human Intelligence is Difficult

(01:43:36) The Lighter Side

The original text contained 11 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

1. The amount of effort is not proportional to the result. One lucid dream (LD) can take hours or even dozens of hours of effort. On average, a practitioner experiences several dozen LDs in their lifetime before quitting. If they don't quit, they dedicate their entire life to it, day and night, trying endless techniques, practicing reality checks, etc.
2. All techniques are absolutely useless compared to the effectiveness of galantamine. The effectiveness of galantamine is directly proportional to its dosage. Thus, 16 mg will almost certainly send you into a powerful lucid dream or out-of-body experience. Therefore, all techniques like WILD are absolutely meaningless.
3. Non-lucid dreams are often more interesting than lucid ones. In a lucid dream, I find myself as my usual self, just in some virtual reality, which is only slightly more interesting than a computer game or watching a movie. But in a non-lucid dream, I [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Here's a new paper on cancer developing in patients after receiving gene therapy with eli-cel, brand name Skysona.

What is eli-cel?

Stem cells are isolated and modified by using a lentiviral vector to add the ABCD1 gene. The stem cells in bone marrow are killed with chemotherapy, and replaced with the modified stem cells.

How often did cancer develop?

Hematologic cancer developed in 7 of 67 patients after the receipt of eli-cel

Yep, that's about what I expected. How long did it take?

myelodysplastic syndrome (MDS) with unilineage dysplasia in 2 patients at 14 and 26 months; MDS with excess blasts in 3 patients at 28, 42, and 92 months; MDS in 1 patient at 36 months; and acute myeloid leukemia (AML) in 1 patient at 57 months.

Makes sense. What does genetic analysis of the cancer cells indicate?

In the [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

tl;dr

Section 1 presents an argument that I’ve heard from a couple people, that says that empathy happens “for free” as a side-effect of the general architecture of mammalian brains, basically because we tend to have similar feelings about similar situations, and “me being happy” is a kinda similar situation to “someone else being happy”, and thus if I find the former motivating then I’ll tend to find the latter motivating too, other things equal.

Section 2 argues that those two situations really aren’t that similar in the grand scheme of things, and that our brains are very much capable of assigning entirely different feelings to pairs of situations even when those situations have some similarities. This happens all the time, and I illustrate my point via the everyday example of having different opinions about tofu versus feta.

Section 3 acknowledges a couple kernels of truth in the Section [...]

---

Outline:

(00:03) tl;dr

(01:01) 1. What am I arguing against?

(05:15) 2. Why I don’t buy it

(05:20) 2.1 Tofu versus feta part 1: the common-sense argument

(07:36) 2.2 Tofu versus feta part 2: The algorithm argument

(10:07) 3. Kernels of truth in the original story

(10:12) 3.1 By default, we can expect transient spillover empathy … before within-lifetime learning promptly eliminates it

(12:10) 3.2 The semantic overlap is stable by default, even if the motivational overlap (from reward model spillover) isn’t

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's monthly roundup time again, and it's happily election-free.

Thinking About the Roman Empire's Approval Rating

Propaganda works, ancient empires edition. This includes the Roman Republic being less popular than the Roman Empire and people approving of Sparta, whereas Persia and Carthage get left behind. They’re no FDA.

Polling USA: Net Favorable Opinion Of:

Ancient Athens: +44%

Roman Empire: +30%

Ancient Sparta: +23%

Roman Republican: +26%

Carthage: +13%

Holy Roman Empire: +7%

Persian Empire: +1%

Visigoths: -7%

Huns: -29%

YouGov / June 6, 2024 / n=2205

The Five Star Problem

What do we do about all 5-star ratings collapsing the way Peter describes here?

Peter Wildeford: TBH I am pretty annoyed that when I rate stuff the options are:

* “5 stars – everything was good enough I guess”

* “4 [...]

---

Outline:

(00:11) Thinking About the Roman Empire's Approval Rating

(01:13) The Five Star Problem

(06:35) Cooking at Home Being Cheaper is Weird

(08:18) With Fans Like These

(09:37) Journalist, Expose Thyself

(13:03) On Not Going the Extra Mile

(13:13) The Rocket Man Said a Bad Bad Thing

(16:27) The Joy of Bad Service

(19:07) Saying What is Not

(19:27) Concentration

(20:26) Should You Do What You Love?

(22:08) Should You Study Philosophy?

(24:31) The Destined Face

(25:09) Tales of Twitter

(34:14) Antisocial Media

(35:01) TikTok On the Clock

(39:07) Tier List of Champions

(40:50) Technology Advances

(42:15) Hotel Hype

(44:44) Government Working

(46:55) I Was Promised Flying Self-Driving Cars

(47:21) For Your Entertainment

(56:50) Cultural Dynamism

(58:43) Hansonian Features

(01:02:19) Variously Effective Altruism

(01:02:45) Nobel Intentions

(01:05:04) Gamers Gonna Game Game Game Game Game

(01:20:17) Sports Go Sports and the Problems with TV Apps These Days

(01:23:46) An Economist Seeks Lunch

(01:30:35) The Lighter Side

The original text contained 6 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

5.1 Post summary / Table of contents

This is the 5th of a series of 8 blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)

Dissociative Identity Disorder (DID) (previously known as “Multiple Personality Disorder”) involves a person having multiple “alters” (alternate identities), with different preferences and (in some cases) different names. A DID diagnosis also requires some nonzero amount of “inter-identity amnesia”, where an alter cannot recall events that occurred when a different alter was active. For example, DSM-V talks about patients “coming to” on a beach with no recollection of how they got there.

Anyway, just like trance in the previous post, DID was one of those things that I unthinkingly assumed was vaguely fictional for most of my life. Maybe that's because my only exposure to it was in actual fiction. While the artistic [...]

---

Outline:

(00:08) 5.1 Post summary / Table of contents

(03:54) 5.2 Why is the suggestion of culture and psychologists a risk factor for DID?

(06:51) 5.3 Why is Borderline Personality Disorder (BPD) a risk factor for DID?

(09:09) 5.3.1 …So the DID alters correspond to different emotional states?

(11:42) 5.4 Why is childhood trauma a risk factor for DID?

(14:34) 5.5 Amnesia

(19:04) 5.5.1 If everyday strong emotions are a bit like mild DID, do they come with mild amnesia??

(21:26) 5.6 Conclusion

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

While you can make a lot of progress in evals with tinkering and paying little attention to the literature, we found that various other papers have saved us many months of research effort. The Apollo Research evals team thus compiled a list of what we felt were important evals-related papers. We likely missed some relevant papers, and our recommendations reflect our personal opinions.

Our favorite papers

• Evaluating Frontier Models for Dangerous Capabilities (Phuong et al., 2024)
  • Contains detailed descriptions of multiple LM agent evals across four categories. Also explores new methodologies for estimating evals success probabilities. 
  • We think it is the best “all around” evals paper, i.e. giving the best understanding of what frontier LM agent evals look like
  • We tested the calibration of their new methodologies in practice in Hojmark et al., 2024, and found that they are not well-calibrated (disclosure: Apollo involvement).
• Observational Scaling Laws and [...]

---

Outline:

(00:29) Our favorite papers

(03:20) Other evals-related publications

(03:25) LM agents

(05:58) Benchmarks

(09:05) Science of evals

(12:02) Software

(13:03) Miscellaneous

(16:49) Related papers from other fields

(16:53) Red teaming

(18:13) Scalable oversight

(19:28) Scaling laws and emergent behaviors

(21:45) Science tutorials

(23:20) LLM capabilities

(25:15) LLM steering

(28:52) Fairness, bias, and accountability

(29:48) AI Governance

(32:31) Contributions

---

First published:

Source:

Narrated by TYPE III AUDIO.