I've noticed that when trying to understand a math paper, there are a few different ways my skill level can be the blocker. Some of these ways line up with some typical levels of organization in math papers:

• Definitions: a formalization of the kind of objects we're even talking about.
• Theorems: propositions on what properties are true of these objects.
• Proofs: demonstrations that the theorems are true of the objects, using known and accepted previous theorems and methods of inference.

Understanding a piece of math will require understanding each of these things in order. It can be very useful to identify which of type of thing I'm stuck on, because the different types can require totally different strategies.

Beyond reading papers, I'm also trying to produce new and useful mathematics. Each of these three levels has another associated skill of generating them. But it seems to me that [...]

---

Outline:

(01:29) Understanding

(01:42) Definitions

(03:28) Theorems

(04:55) Proofs

(07:29) Generating

(07:52) Proofs

(08:58) Theorems

(10:08) Definitions

(12:50) Skill improvement

(14:24) Paradigm formation starts at the end

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Henry George's 1879 book Progress & Poverty was the second best-selling book in the entire world during the 1880s and 1890s, outsold by only the Bible. Nobody knows exactly how many copies it sold during those two decades since nobody was keeping track, but it definitely sold at least several millions of copies for sure. The Progressive Era is literally named after the book itself.

Georgism used to have millions of followers, and many of them were very famous people. When Henry George died in 1897 (just a few days before the election for New York City mayor), an estimated 100,000 people attended this funeral.

The mid-20th century labor economist and journalist George Soule wrote that George was “By far the most famous American economic writer,” and “author of a book which probably had a larger world-wide circulation than any other work on economics ever [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Work done in Neel Nanda's stream of MATS 6.0.

Epistemic status: Tried this on a single sweep and seems to work well, but it might definitely be a fluke of something particular to our implementation or experimental set-up. As there are also some theoretical reasons to expect this technique to work (adaptive sparsity), it seems probable that for many TopK SAE set-ups it could be a good idea to also try BatchTopK. As we’re not planning to investigate this much further and it might be useful to others, we’re just sharing what we’ve found so far.

TL;DR: Instead of taking the TopK feature activations per token during training, taking the Top(K*batch_size) for every batch seems to improve SAE performance. During inference, this activation can be replaced with a single global threshold for all features.

Introduction

Sparse [...]

---

Outline:

(01:05) Introduction

(01:53) BatchTopK

(02:36) Experimental Set-Up

(03:47) Results

(05:39) Inference with BatchTopK

(07:44) Limitations and Future Work

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

New paper from the Google DeepMind mechanistic interpretability team, led by Sen Rajamanoharan!

We introduce JumpReLU SAEs, a new SAE architecture that replaces the standard ReLUs with discontinuous JumpReLU activations, and seems to be (narrowly) state of the art over existing methods like TopK and Gated SAEs for achieving high reconstruction at a given sparsity level, without a hit to interpretability. We train through discontinuity with straight-through estimators, which also let us directly optimise the L0.

To accompany this, we will release the weights of hundreds of JumpReLU SAEs on every layer and sublayer of Gemma 2 2B and 9B in a few weeks. Apply now for early access to the 9B ones! We're keen to get feedback from the community, and to get these into the hands of researchers as fast as [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is an interim report sharing preliminary results that we are currently building on. We hope this update will be useful to related research occurring in parallel.

Executive Summary

• We train SAEs on base / chat model pairs and find that SAEs trained on the base model transfer surprisingly well to reconstructing chat activations (and vice versa) on Mistral-7B and Qwen 1.5 0.5B.
  • We also find that they don’t transfer on Gemma v1 2B, and are generally bad at reconstructing <1% of unusually high norm activations (e.g. BOS tokens) from the opposite model.
• We fine-tune our base Mistral-7B SAE (on 5 million chat activations) to cheaply obtain an SAE with competitive sparsity and reconstruction fidelity to a chat SAE trained from scratch (on 800M tokens).
• We open source base, chat, and fine-tuned SAEs (plus wandb runs) [...]

---

Outline:

(00:22) Executive Summary

(01:46) Introduction

(04:14) Investigating SAE Transfer between base and chat models

(07:51) Identifying failures: Outlier norm activations and Gemma v1 2B

(10:24) Investigating SAE Transfer on Instruction Formatted Data

(12:48) Fine-tuning Base SAEs for Chat Models

(15:31) Conclusion

(16:20) Limitations

(17:57) Citing this work

(18:11) Author contributions Statement

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Adam Mastroianni writes about "actually caring about stuff, and for the right reasons", rather than just LARPing. The opening is excerpted below.

I once saw someone give a talk about a tiny intervention that caused a gigantic effect, something like, “We gave high school seniors a hearty slap on the back and then they scored 500 points higher on the SAT.”1

Everyone in the audience was like, “Hmm, interesting, I wonder if there were any gender effects, etc.”

I wanted to get up and yell: “EITHER THIS IS THE MOST POTENT PSYCHOLOGICAL INTERVENTION EVER, OR THIS STUDY IS TOTAL BULLSHIT.”

If those results are real, we should start a nationwide backslapping campaign immediately. We should be backslapping astronauts before their rocket launches and Olympians before their floor routines. We should be running followup studies to see just how many SAT points we can get—does a [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

What do you call a clause explicitly saying that you waive the right to whistleblower compensation, and that you need to get permission before sharing information with government regulators like the SEC?

I have many answers.

I also know that OpenAI, having f***ed around, seems poised to find out, because that is the claim made by whistleblowers to the SEC. Given the SEC fines you for merely not making an explicit exception to your NDA for whistleblowers, what will they do once aware of explicit clauses going the other way?

(Unless, of course, the complaint is factually wrong, but that seems unlikely.)

We also have rather a lot of tech people coming out in support of Trump. I go into the reasons why, which I do think is worth considering. There is a mix of explanations, and at least one very good reason.

Then [...]

---

Outline:

(01:40) Language Models Offer Mundane Utility

(08:10) Language Models Don’t Offer Mundane Utility

(10:19) Clauding Along

(12:25) Fun with Image Generation

(13:46) Deepfaketown and Botpocalypse Soon

(14:49) They Took Our Jobs

(18:29) Get Involved

(20:14) Introducing

(21:46) In Other AI News

(25:02) Denying the Future

(26:59) Quiet Speculations

(32:34) The Quest for Sane Regulations

(37:03) The Other Quest Regarding Regulations

(57:02) SB 1047 Opposition Watch (1)

(01:07:41) SB 1047 Opposition Watch (2)

(01:11:21) Open Weights are Unsafe and Nothing Can Fix This

(01:12:21) The Week in Audio

(01:14:21) Rhetorical Innovation

(01:14:48) Oh Anthropic

(01:16:57) Openly Evil AI

(01:24:21) Aligning a Smarter Than Human Intelligence is Difficult

(01:37:30) People Are Worried About AI Killing Everyone

(01:37:52) Other People Are Not As Worried About AI Killing Everyone

(01:39:49) The Lighter Side

The original text contained 19 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.