How can we make many humans who are very good at solving difficult problems?

Summary (table of made-up numbers)

I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers.

Call to action

If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would greatly accelerate this technology.

If you're young and smart, or are already an expert in either stem cell / reproductive biology, biotech, or anything related to brain-computer interfaces, there are some projects you could work on.

If neither, think hard, maybe I missed something.

You can DM me or gmail [...]

---

Outline:

(00:12) Summary (table of made-up numbers)

(00:45) Call to action

(01:22) Context

(01:25) The goal

(02:56) Constraint: Algernons law

(04:30) How to know what makes a smart brain

(04:35) Figure it out ourselves

(04:53) Copy natures work

(05:18) Brain emulation

(05:21) The approach

(06:07) Problems

(07:52) Genomic approaches

(08:34) Adult brain gene editing

(08:38) The approach

(08:53) Problems

(09:26) Germline engineering

(09:32) The approach

(11:37) Problems

(12:11) Signaling molecules for creative brains

(12:15) The approach

(13:30) Problems

(13:45) Brain-brain electrical interface approaches

(14:41) Problems with all electrical brain interface approaches

(15:11) Massive cerebral prosthetic connectivity

(17:03) Human / human interface

(17:59) Interface with brain tissue in a vat

(18:30) Massive neural transplantation

(18:35) The approach

(19:01) Problems

(19:39) Support for thinking

(19:53) The approaches

(21:04) Problems

(21:58) FAQ

(22:01) What about weak amplification

(22:14) What about ...

(24:04) The real intelligence enhancement is ...

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If I say "the store is 500 meters away," is this strictly true? Strictly false? Either strictly true or strictly false, with probabilies of true or false summing to one? Fuzzily true, because the store is 500.1 meters away? My thesis is that it's strictly true or strictly false, with associated probabilities. Bear with me.


The 500 meter example is pretty hard to reason about numerically. I am hopeful that I can communicate my thoughts by starting with an example that is in some sense simpler.

Is our conversational partner making a true statement or a false statement? What we're used to is starting with a prior over worlds, and then updating our belief based on observations- the man is underwater, the man is currently moving downwards. So our belief that the man is sinking should be pretty high. [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I interact with journalists quite a lot and I have specific preferences. Not just for articles, but for behaviour. And journalists do behave pretty strangely at times.

This account comes from talking to journalists on ~10 occasions. Including being quoted in ~5 articles.

Privacy

I do not trust journalists to abide by norms of privacy. If I talk to a friend and without asking, share what they said, with their name attached, I expect they'd be upset. But journalists regularly act as if their profession sets up the opposite norm - that everything is publishable, unless explicitly agreed otherwise. This is bizarre to me. It's like they have taken a public oath to be untrustworthy.

Perhaps they would argue that it's a few bad journalists who behave like this, but how am I supposed to know which the good and bad journalists are? The general [...]

---

Outline:

(00:24) Privacy

(02:41) Preserving meaning

(04:42) Reporters vs pundits

(09:18) Accuracy

(14:16) My personal stance

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Previously: https://www.lesswrong.com/posts/RydETq379eoWqBFvj/updates-and-reflections-on-optimal-exercise-after-nearly-a

https://www.lesswrong.com/posts/bZ2w99pEAeAbKnKqo/optimal-exercise

I figured it would be useful to summarize how I train when I go for time efficiency these days, based on my understanding of the relevant meta-analyses on exercise selection and parameters. The primary time savers are supersets and lowered rest times. A routine like this is not totally optimized for max hypertrophy or anything, but is very very time efficient, and would still likely cause large gains for relatively untrained people. A lot of why I exercise these days, besides maintenance, is energy levels and injury prevention.

With supersets, you spend the time resting one group of muscles by doing an exercise for a different group of muscles. This is well supported as having the same or even slightly greater improvements in exercise studies, with the main downside being that the overall greater work packed in to a smaller amount of time being somewhat more [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

As part of our Summer 2024 Program, MATS ran a series of discussion groups focused on questions and topics we believe are relevant to prioritizing research into AI safety. Each weekly session focused on one overarching question, and was accompanied by readings and suggested discussion questions. The purpose of running these discussions was to increase scholars’ knowledge about the AI safety ecosystem and models of how AI could cause a catastrophe, and hone scholars’ ability to think critically about threat models—ultimately, in service of helping scholars become excellent researchers.

The readings and questions were largely based on the curriculum from the Winter 2023-24 Program, with two changes:

• We reduced the number of weeks, since in the previous cohort scholars found it harder to devote time to discussion groups later in the program.
• For each week we selected a small number of “core readings”, since many scholars were unable [...]

---

Outline:

(01:57) Week 1: How powerful is intelligence?

(02:03) Core readings

(02:37) Other readings

(03:34) Discussion questions

(04:50) Week 2: How and when will transformative AI be made?

(04:56) Core readings

(07:06) Other readings

(09:23) Discussion questions

(10:30) Week 3: How could we train AIs whose outputs we can’t evaluate?

(10:37) Core readings

(12:00) Other readings

(15:01) Discussion questions

(16:01) Week 4: Will AIs fake alignment?

(16:06) Core readings

(16:25) Other readings

(16:28) On inner and outer alignment

(17:15) On reasons to think deceptive alignment is likely

(18:07) Discussion questions

(19:25) Week 5: How should AI be governed?

(19:31) Core readings

(20:49) Other readings

(24:10) Discussion questions

(25:32) Readings that did not fit into any specific week

(26:26) Acknowledgements

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 629 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Last week, ARC released a paper called Towards a Law of Iterated Expectations for Heuristic Estimators, which follows up on previous work on formalizing the presumption of independence. Most of the work described here was done in 2023.

A brief table of contents for this post:

• What is a heuristic estimator? (One example and three analogies.)
• How might heuristic estimators help with understanding neural networks? (Three potential applications.)
• Formalizing the principle of unpredictable errors for heuristic estimation (the technical meat of the paper).

In "Formalizing the Presumption of Independence", we defined a heuristic estimator to be a hypothetical algorithm that estimates the values of mathematical expression based on arguments. That is, a heuristic estimator is [...]

---

Outline:

(04:56) What is a heuristic estimator?

(05:31) Example: Sum of sixth digits of square roots

(07:11) Analogy #1: Proof verification

(08:36) Analogy #2: Conditional expectation

(09:38) Analogy #3: Subjective probabilities and estimates

(11:43) How can heuristic estimation help us understand neural networks?

(14:16) Mechanistic anomaly detection

(16:33) Safe distillation

(19:28) Low probability estimation

(21:37) Formalizing the principle of unpredictable errors

(22:50) The subjective approach: Iterated estimation and error orthogonality

(26:37) Challenges with the subjective approach

(28:43) The objective approach: Accuracy

(35:28) Challenges with the objective approach

(36:14) Estimating the product of jointly normal random variables

(41:07) Estimating the permanent of a matrix

(44:21) Conclusion

The original text contained 19 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TLDR;

In previous work, we found a problematic form a feature splitting called "feature absorption" when analyzing Gemma Scope SAEs. We hypothesized that this was due to SAEs struggling to separate co-occurrence between features, but we did not prove this. In this post, we set up toy models where we can explicitly control feature representations and co-occurrence rates and show the following:

• Feature absorption happens when features co-occur.
• If co-occurring feature magnitudes vary relative to each other, we observe "partial absorption", where a latent tracking a main feature sometimes fires weakly instead of not firing at all, but sometimes does fully not fire.
• Feature absorption happens even with imperfect co-occurrence, depending on the strength of the sparsity penalty.
• Tying the SAE encoder and decoder weights together solves feature absorption.

All code for this post can be seen in this Colab notebook.

The rest of this post will assume [...]

---

Outline:

(00:06) TLDR;

(01:18) What is feature absorption?

(02:15) How is this different than traditional feature splitting?

(03:47) Why does absorption happen?

(04:17) How big of a problem is this, really?

(05:04) Toy Models of Feature Absorption Setup

(05:29) Non-superposition setup

(07:12) Superposition setup

(07:53) Perfect Reconstruction with Independent Features

(08:39) Feature co-occurrence causes absorption

(10:33) Magnitude variance causes partial absorption

(12:35) Why does partial absorption happen?

(13:13) Imperfect co-occurrence can still lead to absorption depending on L1 penalty

(16:03) Tying the SAE encoder and decoder weights solves feature absorption

(17:08) Absorption in superposition

(19:13) Tying the encoder and decoder weights still solves feature absorption in superposition.

(19:53) Future work

The original text contained 10 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: This essay attempts to communicate intuitions strong shaped by @Joe Carlsmith's writing, particularly "On Green."

Warning: very minor spoilers, mostly for "The Dark Knight".

Some people have a sense of weight about them. They know exactly what they want and can explain why it is Right. Presented with a moral dilemma, they resolve conflicts between local values in a consistent way by appealing to higher values - they can tell you today whether they would actually pull the lever in the trolley problem and why they (hopefully) wouldn't harvest the organs from one patient to save five - or why they would be compelled to. Most of these people seem to be deeply religious - the real-world example I have in mind is an intelligent Catholic friend who once had his bike stolen and explained to me that it was a shame because he would have happily [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

We have published A Narrow Path: our best attempt to draw out a comprehensive plan to deal with AI extinction risk. We propose concrete conditions that must be satisfied for addressing AI extinction risk, and offer policies that enforce these conditions.

A Narrow Path answers the following: assuming extinction risk from AI, what would be a response that actually solves the problem for at least 20 years, and that leads to a stable global situation, one where the response is coordinated rather than unilaterally imposed with all the dangers that come from that.

Despite the magnitude of the problem, we have found no other plan that comprehensively tries to address the issue, so we made one.

This is a complex problem where no one has a full solution, but we need to iterate on better answers if we are to succeed at implementing solutions [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Damon Sasi (@DaystarEld) is a mentor of mine, and he is also a therapist who claims to be very psychologically healthy. What's his inner experience like?

• “Probably for an average week, maybe I'll suffer for like five minutes”
• “I don't think I have any prolonged internal conflict”
• “Self-loathing to me was a weird concept […] Like, what does it mean to like self-loathe?”

It's not that he had a great childhood, either.

[My childhood] wasn't terrible. And I say this and then maybe I'll give some examples and people will be like, maybe it was terrible. […] my parents divorced when I was pretty young. Both my parents, I think, were very absent. […]

He would get drunk often. He once hit me in the back with a two by four. My older brother was pretty physically abusive before he chilled out. And by chilled out, I mean [...]

---

Outline:

(00:33) It's not that he had a great childhood, either.

(01:35) What is it like to be psychologically healthy?

(02:45) Details in podcast.

---

First published:

Source:

Narrated by TYPE III AUDIO.