The AGI Safety & Alignment Team (ASAT) at Google DeepMind (GDM) is hiring! Please apply to the Research Scientist and Research Engineer roles. Strong software engineers with some ML background should also apply (to the Research Engineer role). Our initial batch of hiring will focus more on hiring engineers, but we expect to continue to use the applications we receive for future hiring this year, which we expect will be more evenly split. Please do apply even if e.g. you’re only available in the later half of this year.

What is ASAT?

ASAT is the primary team at GDM focused on technical approaches to severe harms from AI systems, having evolved out of the Alignment and Scalable Alignment teams. We’re organized around two themes: AGI Alignment (think amplified oversight and mechanistic interpretability) and Frontier Safety (think development and implementation of the Frontier Safety Framework). The leadership team is Anca [...]

---

Outline:

(00:40) What is ASAT?

(01:14) Why should you join?

(03:05) What will we do in the near future?

(05:37) How do you prioritize across research topics?

(09:20) FAQ

(15:44) Apply now!

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is an all-in-one crosspost of a scenario I originally published in three parts on my blog (No Set Gauge). Links to the originals:

• A History of the Future, 2025-2027
• A History of the Future, 2027-2030
• A History of the Future, 2030-2040

Thanks to Luke Drago, Duncan McClements, and Theo Horsley for comments on all three parts.

2025-2027

Below is part 1 of an extended scenario describing how the future might go if current trends in AI continue. The scenario is deliberately extremely specific: it's definite rather than indefinite, and makes concrete guesses instead of settling for banal generalities or abstract descriptions of trends.

The return of reinforcement learning

From 2019 to 2023, the main driver of AI was using more compute and data for pretraining. This was combined with some important "unhobblings":

• Post-training (supervised fine-tuning and reinforcement learning for [...]

---

Outline:

(00:34) 2025-2027

(01:04) The return of reinforcement learning

(10:52) Codegen, Big Tech, and the internet

(21:07) Business strategy in 2025 and 2026

(27:23) Maths and the hard sciences

(33:59) Societal response

(37:18) Alignment research and AI-run orgs

(44:49) Government wakeup

(51:42) 2027-2030

(51:53) The AGI frog is getting boiled

(01:02:18) The bitter law of business

(01:06:52) The early days of the robot race

(01:10:12) The digital wonderland, social movements, and the AI cults

(01:24:09) AGI politics and the chip supply chain

(01:33:04) 2030-2040

(01:33:15) The end of white-collar work and the new job scene

(01:47:47) Lab strategy amid superintelligence and robotics

(01:56:28) Towards the automated robot economy

(02:15:49) The human condition in the 2030s

(02:17:26) 2040+

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Direct PDF link for non-subscribers

Information theory must precede probability theory, and not be based on it. By the
very essence of this discipline, the foundations of information theory have a finite combinatorial character.

- Andrey Kolmogorov

Many alignment researchers borrow intuitions from thermodynamics: entropy relates to information, which relates to learning and epistemology. These connections were first revealed by Szilárd's resolution of Maxwell's famous thought experiment. However, the classical tools of equilibrium thermodynamics are not ideally suited to studying information processing far from equilibrium.

This new work reframes thermodynamics in terms of the algorithmic entropy. It takes an information-first approach, delaying the introduction of physical concepts such as heat and energy until after the foundations are set. I find this approach more conceptually principled and elegant than the traditional alternatives.

It's based on a 30-year-old workshop paper by Péter Gács[1], which until now was [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I recently posted about doing Celtic Knots on a Hexagonal lattice ( https://www.lesswrong.com/posts/tgi3iBTKk4YfBQxGH/celtic-knots-on-a-hex-lattice ).

There were many nice suggestions in the comments. @Shankar Sivarajan suggested that I could look at a Einstien lattice instead, which sounded especially interesting. ( https://en.wikipedia.org/wiki/Einstein_problem . )

The idea of the Einstein tile is that it can tile the plane (like a hexagon or square can), but it does so in a way where the pattern of tiles never repeats.

The tile I took from Wikipedia looks like this:

On the left is the full tile. On the right is a way of decomposing it into four-thirds of a hexagon.

For some reason I think of it as a lama. On the left top is its head, facing left. On tie right top its its tail. The squarish bit coming down is the legs.

First problem: the tile has 13 sides. So if [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm planning to organize a mentorship programme for people who want to become researchers working on the Learning-Theoretic Agenda (LTA). I'm still figuring out the detailed plan, the logistics and the funding, but here's an outline of how it would looks like. To express interest, submit this form.

Why Apply?

I believe that the risk of a global catastrophe due to unaligned artificial superintelligence is the most pressing problem of our time. I also believe that the LTA is our last best chance to solving the technical alignment problem. Even if governance efforts succeed, they will only buy time: but we still need to use this time somehow. The LTA is how we use this time. (I also consider some other research agendas to be useful, but mostly inasmuch as they can be combined with LTA.)

Fortunately, the LTA has many shovel-ready research directions that can be advanced in [...]

---

Outline:

(00:24) Why Apply?

(01:16) Requirements

(02:11) Application Process

(02:44) Programme Structure

(03:36) Programme Content

---

First published:

Source:

Narrated by TYPE III AUDIO.

On March 14th, 2015, Harry Potter and the Methods of Rationality made its final post. Wrap parties were held all across the world to read the ending and talk about the story, in some cases sparking groups that would continue to meet for years. It's been ten years, and think that's a good reason for a round of parties.

If you were there a decade ago, maybe gather your friends and talk about how things have changed. If you found HPMOR recently and you're excited about it (surveys suggest it's still the biggest on-ramp to the community, so you're not alone!) this is an excellent chance to meet some other fans in person for the first time!

Want to run an HPMOR Anniversary Party, or get notified if one's happening near you? Fill out this form.

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Microplastics have been in the news and the local rationalist discord with increasing frequency over the past few months. Of note are Nat Friedman's PlasticList, and this 2024 preprint demonstrating that some human brains are now 0.5% plastic by weight. There was also an article on black plastic cookware that was quickly retracted[1]. The volume of discussion made it clear that few people seem to have a clear understanding of what we actually know about microplastics, so we had a meetup about it.

Here are the notes from Kitchener-Waterloo Rationality's in-person meetup[2] researching the current understanding of microplastics in a time-boxed format. Sort of like partitioned book club, the goal of the meetup was to power of friendship teamwork our way into a better understanding of a complex topic with relatively little individual effort.

tl;dr: The field is so new that there's a striking lack of scientific consensus [...]

---

Outline:

(01:59) What are the current recommended limits for microplastic intake?

(02:06) Canada

(02:44) EU

(02:56) USA

(05:42) World Health Organization

(06:25) Recommended Limits For Specific Chemicals Exist

(07:34) A Good Paper on Microplastic Regulation Meta

(09:15) Health Effects of Microplastics

(12:33) Quantities Ingested and Main Exposure Pathways

(18:06) Food contamination

(20:49) Household dust

(23:19) How to reduce the amount of microplastics you have

(23:32) Misc links

(26:21) Appendix: Research Agenda and Format

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 134 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

In a recent paper in Annals of Mathematics and Philosophy, Fields medalist Timothy Gowers asks why mathematicians sometimes believe that unproved statements are likely to be true. For example, it is unknown whether _pi_ is a normal number (which, roughly speaking, means that every digit appears in _pi_ with equal frequency), yet this is widely believed. Gowers proposes that there is no sign of any reason for _pi_ to be non-normal -- especially not one that would fail to reveal itself in the first million digits -- and in the absence of any such reason, any deviation from normality would be an outrageous coincidence. Thus, the likely normality of _pi_ is inferred from the following general principle:

No-coincidence [...]

---

Outline:

(02:32) Our no-coincidence conjecture

(05:37) How we came up with the statement

(08:31) Thoughts for theoretical computer scientists

(10:27) Why we care

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

We are excited to release a short course on AGI safety for students, researchers and professionals interested in this topic. The course offers a concise and accessible introduction to AI alignment, consisting of short recorded talks and exercises (75 minutes total) with an accompanying slide deck and exercise workbook. It covers alignment problems we can expect as AI capabilities advance, and our current approach to these problems (on technical and governance levels). If you would like to learn more about AGI safety but have only an hour to spare, this course is for you!

Here are some key topics you will learn about in this course:

• The evidence for the field being on a path to advanced AI capabilities.
• Arguments for instrumental subgoals and deliberate planning towards a misaligned goal as a source of extreme risk.
• Two ways in which misaligned goals may [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This post covers three recent shenanigans involving OpenAI.

In each of them, OpenAI or Sam Altman attempt to hide the central thing going on.

First, in Three Observations, Sam Altman's essay pitches our glorious AI future while attempting to pretend the downsides and dangers don’t exist in some places, and in others admitting we’re not going to like those downsides and dangers but he's not about to let that stop him. He's going to transform the world whether we like it or not.

Second, we have Frog and Toad, or There Is No Plan, where OpenAI reveals that its plan for ensuring AIs complement humans rather than AIs substituting for humans is to treat this as a ‘design choice.’ They can simply not design AIs that will be substitutes. Except of course this is Obvious Nonsense in context, with all the talk of remote workers, and [...]

---

Outline:

(01:52) Three Observations

(11:38) Frog and Toad (or There Is No Plan)

(18:45) A Trade Offer Has Arrived

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.