Direct PDF link for non-subscribers

Information theory must precede probability theory, and not be based on it. By the
very essence of this discipline, the foundations of information theory have a finite combinatorial character.

- Andrey Kolmogorov

Many alignment researchers borrow intuitions from thermodynamics: entropy relates to information, which relates to learning and epistemology. These connections were first revealed by Szilárd's resolution of Maxwell's famous thought experiment. However, the classical tools of equilibrium thermodynamics are not ideally suited to studying information processing far from equilibrium.

This new work reframes thermodynamics in terms of the algorithmic entropy. It takes an information-first approach, delaying the introduction of physical concepts such as heat and energy until after the foundations are set. I find this approach more conceptually principled and elegant than the traditional alternatives.

It's based on a 30-year-old workshop paper by Péter Gács[1], which until now was [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I recently posted about doing Celtic Knots on a Hexagonal lattice ( https://www.lesswrong.com/posts/tgi3iBTKk4YfBQxGH/celtic-knots-on-a-hex-lattice ).

There were many nice suggestions in the comments. @Shankar Sivarajan suggested that I could look at a Einstien lattice instead, which sounded especially interesting. ( https://en.wikipedia.org/wiki/Einstein_problem . )

The idea of the Einstein tile is that it can tile the plane (like a hexagon or square can), but it does so in a way where the pattern of tiles never repeats.

The tile I took from Wikipedia looks like this:

On the left is the full tile. On the right is a way of decomposing it into four-thirds of a hexagon.

For some reason I think of it as a lama. On the left top is its head, facing left. On tie right top its its tail. The squarish bit coming down is the legs.

First problem: the tile has 13 sides. So if [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm planning to organize a mentorship programme for people who want to become researchers working on the Learning-Theoretic Agenda (LTA). I'm still figuring out the detailed plan, the logistics and the funding, but here's an outline of how it would looks like. To express interest, submit this form.

Why Apply?

I believe that the risk of a global catastrophe due to unaligned artificial superintelligence is the most pressing problem of our time. I also believe that the LTA is our last best chance to solving the technical alignment problem. Even if governance efforts succeed, they will only buy time: but we still need to use this time somehow. The LTA is how we use this time. (I also consider some other research agendas to be useful, but mostly inasmuch as they can be combined with LTA.)

Fortunately, the LTA has many shovel-ready research directions that can be advanced in [...]

---

Outline:

(00:24) Why Apply?

(01:16) Requirements

(02:11) Application Process

(02:44) Programme Structure

(03:36) Programme Content

---

First published:

Source:

Narrated by TYPE III AUDIO.

On March 14th, 2015, Harry Potter and the Methods of Rationality made its final post. Wrap parties were held all across the world to read the ending and talk about the story, in some cases sparking groups that would continue to meet for years. It's been ten years, and think that's a good reason for a round of parties.

If you were there a decade ago, maybe gather your friends and talk about how things have changed. If you found HPMOR recently and you're excited about it (surveys suggest it's still the biggest on-ramp to the community, so you're not alone!) this is an excellent chance to meet some other fans in person for the first time!

Want to run an HPMOR Anniversary Party, or get notified if one's happening near you? Fill out this form.

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Microplastics have been in the news and the local rationalist discord with increasing frequency over the past few months. Of note are Nat Friedman's PlasticList, and this 2024 preprint demonstrating that some human brains are now 0.5% plastic by weight. There was also an article on black plastic cookware that was quickly retracted[1]. The volume of discussion made it clear that few people seem to have a clear understanding of what we actually know about microplastics, so we had a meetup about it.

Here are the notes from Kitchener-Waterloo Rationality's in-person meetup[2] researching the current understanding of microplastics in a time-boxed format. Sort of like partitioned book club, the goal of the meetup was to power of friendship teamwork our way into a better understanding of a complex topic with relatively little individual effort.

tl;dr: The field is so new that there's a striking lack of scientific consensus [...]

---

Outline:

(01:59) What are the current recommended limits for microplastic intake?

(02:06) Canada

(02:44) EU

(02:56) USA

(05:42) World Health Organization

(06:25) Recommended Limits For Specific Chemicals Exist

(07:34) A Good Paper on Microplastic Regulation Meta

(09:15) Health Effects of Microplastics

(12:33) Quantities Ingested and Main Exposure Pathways

(18:06) Food contamination

(20:49) Household dust

(23:19) How to reduce the amount of microplastics you have

(23:32) Misc links

(26:21) Appendix: Research Agenda and Format

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 134 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

In a recent paper in Annals of Mathematics and Philosophy, Fields medalist Timothy Gowers asks why mathematicians sometimes believe that unproved statements are likely to be true. For example, it is unknown whether _pi_ is a normal number (which, roughly speaking, means that every digit appears in _pi_ with equal frequency), yet this is widely believed. Gowers proposes that there is no sign of any reason for _pi_ to be non-normal -- especially not one that would fail to reveal itself in the first million digits -- and in the absence of any such reason, any deviation from normality would be an outrageous coincidence. Thus, the likely normality of _pi_ is inferred from the following general principle:

No-coincidence [...]

---

Outline:

(02:32) Our no-coincidence conjecture

(05:37) How we came up with the statement

(08:31) Thoughts for theoretical computer scientists

(10:27) Why we care

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

We are excited to release a short course on AGI safety for students, researchers and professionals interested in this topic. The course offers a concise and accessible introduction to AI alignment, consisting of short recorded talks and exercises (75 minutes total) with an accompanying slide deck and exercise workbook. It covers alignment problems we can expect as AI capabilities advance, and our current approach to these problems (on technical and governance levels). If you would like to learn more about AGI safety but have only an hour to spare, this course is for you!

Here are some key topics you will learn about in this course:

• The evidence for the field being on a path to advanced AI capabilities.
• Arguments for instrumental subgoals and deliberate planning towards a misaligned goal as a source of extreme risk.
• Two ways in which misaligned goals may [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This post covers three recent shenanigans involving OpenAI.

In each of them, OpenAI or Sam Altman attempt to hide the central thing going on.

First, in Three Observations, Sam Altman's essay pitches our glorious AI future while attempting to pretend the downsides and dangers don’t exist in some places, and in others admitting we’re not going to like those downsides and dangers but he's not about to let that stop him. He's going to transform the world whether we like it or not.

Second, we have Frog and Toad, or There Is No Plan, where OpenAI reveals that its plan for ensuring AIs complement humans rather than AIs substituting for humans is to treat this as a ‘design choice.’ They can simply not design AIs that will be substitutes. Except of course this is Obvious Nonsense in context, with all the talk of remote workers, and [...]

---

Outline:

(01:52) Three Observations

(11:38) Frog and Toad (or There Is No Plan)

(18:45) A Trade Offer Has Arrived

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 136 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Introduction

It's now well known that simple neural network models often "grok" algorithmic tasks. That is, when trained for many epochs on a subset of the full input space, the model quickly attains perfect train accuracy and then, much later, near-perfect test accuracy. In the former phase, the model memorizes the training set; in the latter, it generalizes out-of-distribution to the test set.

In the algorithmic grokking literature, there is typically exactly one natural generalization from the training set to the test set. What if, however, the training set were instead under-specified in such a way that there were multiple possible generalizations? Would the model grok at all? If so, which of the generalizing solutions would it choose? [...]

---

Outline:

(00:20) Introduction

(01:21) Setup

(03:45) Experiments

(03:48) Ambiguous grokking

(03:52) Grokking either group

(04:54) Grokking the intersect

(05:48) Grokking only one group

(06:41) No grokking

(07:13) Measuring complexity

(07:16) Complexity of the grokked solution

(11:53) Complexity over time

(13:57) Determination and differentiation

(14:01) Perturbation sensitivity

(15:47) Total variation

(17:14) Determination across distribution shift

(18:32) Training Jacobian

(21:03) Discussion

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 32 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

[Thanks to Steven Byrnes for feedback and the idea for section §3.1. Also thanks to Justice from the LW feedback team.]

Remember this?

Or this?

The images are from WaitButWhy, but the idea was voiced by many prominent alignment people, including Eliezer Yudkowsky and Nick Bostrom. The argument is that the difference in brain architecture between the dumbest and smartest human is so small that the step from subhuman to superhuman AI should go extremely quickly. This idea was very pervasive at the time. It's also wrong. I don't think most people on LessWrong have a good model of why it's wrong, and [...]

---

Outline:

(02:07) 1. Why Village Idiot to Einstein is a Long Road: The Two-Component Model of Intelligence

(06:04) 2. Sequential Reasoning

(07:03) 2.1 Sequential Reasoning without a Thought Assessor: the Story so Far

(13:03) 2.2 Sequential Reasoning without a Thought Assessor: the Future

(16:29) 3. More Thoughts on Sequential Reasoning

(16:33) 3.1. Subdividing the Category

(21:09) 3.2. Philosophical Intelligence

(27:17) 4. Miscellaneous

(27:21) 4.1. Your description of thought generation and assessment seems vague, underspecified, and insufficiently justified to base your conclusions on.

(28:08) 4.2. If your model is on the right track, doesnt this mean that human+AI teams could achieve significantly superhuman performance and that these could keep outperforming pure AI systems for a long time?

(29:48) 4.3. What does this mean for safety, p(doom), etc.?

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.