A man has just robbed a train. A Rock Island Rail car was held up in the desert of North Texas on route from Chicago carrying dozens of passengers and tens of thousands of dollars in a Wells Fargo express compartment. The men were slain and the women and children are kidnapped.

You’re the sheriff in charge of the investigation. In a case like this, tried and true procedure is to put a bounty on the man's head. That gets everyone interested in looking and bringing this man to justice. An everyday 2-bit robber might get a bounty of a thousand or ten, but this is an emergency. You set the bounty at $200,000, dead or alive.

This bandit ain’t getting away easy.

With two major hurricanes in the last couple of weeks, “price gouging” is in the news. Whether it's $10 for a gallon [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Summary

I make a large part of my living performing literature reviews to answer scientific questions. For years AI has been unable to do anything to lower my research workload, but back in August I tried Perplexity, and it immediately provided value far beyond what I’d gotten from other tools. This wasn’t a fair comparison because I hadn’t tried any other AI research assistant in months, which is decades in AI time. In this post I right that wrong by running two test questions through every major tool, plus a smaller tool recommended in the comments of the last post

Spoilers: the result was a rough tie between You.com and Elicit. Each placed first on one task and was among top-3 in the other.

Tasks + Results

Tl;dr:

---

Outline:

(00:08) Summary

(00:55) Tasks + Results

(01:37) Finding papers on water gargling as an antiviral

(03:50) Hepcidin

(04:57) (Lack of) Conflict of interest

(05:30) Conclusions

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

An OODA loop is a repeating loop where you take in information, integrate it into your decision-making, use it to make more decisions, and then receive more information in response.

“OODA” is a particular structure referring to:

• Observe: “what is happening in the world” (in as raw a form as you can)
• Orient: “what sorts of things are important right now? How do I organize the information I’m seeing”
• Decide: “what do I want to do next”
• Act: “Okay I’m doing it now.”

You are probably doing some kind of OODA-loop-like-thing already. But it's worth reflecting on how you could do it better.

For my last workshop, partly because it was cute, and partly because it seemed like probably a good idea, I put together a worksheet for applying a deliberate, intentional OODA loop to your OODA loop.

I don't actually know if this worksheet turned out to be helpful (it [...]

---

Outline:

(01:27) Observation

(02:13) Write details about some short-timeframe examples of…

(02:18) Observing

(02:21) Orienting

(02:25) Deciding

(02:28) Acting

(02:39) Write details about some longer-timeframe examples of…

(02:44) Observing

(02:47) Orienting

(02:51) Deciding

(02:54) Acting

(02:57) Orient

(03:29) What seems important to track about OODA loops?

(03:34) Write a short 1-2 paragraphs or bullet list describing your high level OODA loop processes.

(03:49) How would you know if you had a good one?

(03:53) What (if anything) are you dissatisfied with about your current OODA loops?

(04:00) What are some ways your OODA loops could be better? (including quick fixes and deeper effortful fixes)

(04:09) What's hard or confusing about improving your OODA loop?

(04:15) Deciding

(04:57) List of plans or actions

(05:01) For each “plan”, what's a next-action you could take right now, to get the ball rolling on it?

(05:08) Act

---

First published:

Source:

Narrated by TYPE III AUDIO.

Is this market really only at 63%? I think you should take the over.

Five tiers of rigor for safety-oriented interpretability work

Lately, I have been thinking of interpretability research as falling into five different tiers of rigor.

1. Pontification

This is when researchers claim they have succeeded in interpreting a model by definition or based on analyzing results and asserting hypotheses about them. This is a key part of the scientific method. But by itself, it is not good science. Previously in this sequence, I have argued that this standard is fairly pervasive.

2. Basic Science

This is when researchers develop an interpretation, use it to make some (usually simple) prediction, and then show that this prediction validates. This is at least doing science, but it doesn't necessarily demonstrate any usefulness or value.

3. Streetlight/Toy Demos

This is [...]

---

Outline:

(00:21) Five tiers of rigor for safety-oriented interpretability work

(00:33) 1. Pontification

(00:58) 2. Basic Science

(01:16) 3. Streetlight/Toy Demos

(01:30) 4. Useful Engineering

(01:55) 5. Net Safety Benefit

(02:24) What's been happening lately?

(02:28) Recently, some solid work has been done in tier 3.

(04:28) I think that tier 4 has (barely) been broken into.

(05:08) Current efforts may soon break further into tier 4.

(06:12) What might happen next with SAEs

(06:16) Past predictions

(08:38) New predictions

(11:41) What if we succeed?

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I think and talk a lot about the risks of powerful AI. The company I’m the CEO of, Anthropic, does a lot of research on how to reduce these risks. Because of this, people sometimes draw the conclusion that I’m a pessimist or “doomer” who thinks AI will be mostly bad or dangerous. I don’t think that at all. In fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us and what I see as a fundamentally positive future. I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be.

In this essay I try to sketch out what that upside might look like—what a world with powerful AI might look like if everything goes right. Of [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This post starts out pretty gloomy but ends up with some points that I feel pretty positive about. Day to day, I'm more focussed on the positive points, but awareness of the negative has been crucial to forming my priorities, so I'm going to start with those. It's mostly addressed to the EA community, but is hopefully somewhat of interest to LessWrong and the Alignment Forum as well.

My main concerns

I think AGI is going to be developed soon, and quickly. Possibly (20%) that's next year, and most likely (80%) before the end of 2029. These are not things you need to believe for yourself in order to understand my view, so no worries if you're not personally convinced of this.

(For what it's worth, I did arrive at this view through years of study and research in AI, combined with over a decade of private forecasting practice [...]

---

Outline:

(00:28) My main concerns

(03:41) Extinction by industrial dehumanization

(06:00) Successionism as a driver of industrial dehumanization

(11:08) My theory of change: confronting successionism with human-specific industries

(15:53) How I identified healthcare as the industry most relevant to caring for humans

(20:00) But why not just do safety work with big AI labs or governments?

(23:22) Conclusion

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Both Geoffrey Hinton and Demis Hassabis were given the Nobel Prize this week, in Physics and Chemistry respectively. Congratulations to both of them along with all the other winners. AI will be central to more and more of scientific progress over time. This felt early, but not as early as you would think.

The two big capability announcements this week were OpenAI's canvas, their answer to Anthropic's artifacts to allow you to work on documents or code outside of the chat window in a way that seems very useful, and Meta announcing a new video generation model with various cool features, that they’re wisely not releasing just yet.

I also have two related corrections from last week, and an apology: Joshua Achiam is OpenAI's new head of Mission Alignment, not of Alignment as I incorrectly said. The new head of Alignment Research is Mia Glaese. That mistake [...]

---

Outline:

(01:30) Language Models Offer Mundane Utility

(09:10) Language Models Don’t Offer Mundane Utility

(13:11) Blank Canvas

(17:13) Meta Video

(18:58) Deepfaketown and Botpocalypse Soon

(21:22) They Took Our Jobs

(24:45) Get Involved

(26:01) Introducing

(26:14) AI Wins the Nobel Prize

(28:51) In Other AI News

(30:05) Quiet Speculations

(34:22) The Mask Comes Off

(37:17) The Quest for Sane Regulations

(41:02) The Week in Audio

(43:13) Rhetorical Innovation

(48:20) The Carbon Question

(50:27) Aligning a Smarter Than Human Intelligence is Difficult

(55:48) People Are Trying Not to Die

The original text contained 6 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post is broken down into two parts:

1. Which AI productivity tools am I currently using (as an alignment researcher)?
2. Why does it currently feel hard to spend +$1000/month on AI to increase one's productivity drastically?

Which AI productivity tools am I currently using?

Let's get right to it. Here's what I'm currently using and how much I am paying:

• Superwhisper (or other new Speech-to-Text that leverage LLMs for rewriting) apps. Under $8.49 per month. You can use different STT models (different speed and accuracy for each) and LLM for rewriting the transcript based on a prompt you give the models. You can also have different "modes," meaning that you can have the model take your transcript and write code instructions in a pre-defined format when you are in an IDE, turn a transcript into a report when writing in Google Docs, etc. I tend to use it [...]

---

Outline:

(00:34) Which AI productivity tools am I currently using?

(08:35) How could I spend way more on AI?

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Once upon a time, there were posts where people would comment with quotes. The last one was a while ago, and they'd been getting less and less traffic.

I have a theory that there's a quote overhang. Firstly, I think LessWrong has recovered a lot since the 2015~2017 decline, and so there might just be more users around. Second, it's been half a decade and so many new quotable lines may have been written. I have a another theory that the way LessWrong 2.0 splits Frontpage and Personal is rougher on a quotes thread than 1.0's Main and Discussion. The best thing to do with a theory is to test it.

The rules are as follows:

• Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Metaculus's Q4 Benchmark series is now live. Click here to learn more and to compete for $30,000. Benchmark the state of the art in AI forecasting against the best humans on real-world questions.

Metaculus's Q3 AI Benchmarking Series aimed to assess how the best bots compare to the best humans on real-world forecasting questions, like those found on Metaculus. Over the quarter we had 55 bots compete for $30,000 on 255 weighted questions with a team of 10 Pros serving as a human benchmark. We found that Pro forecasters were significantly better than top bots (p = 0.036) using log scoring with a weighted t-test.

This main result compares the median forecast of 10 Pro Forecasters against the median forecast of 9 top bots on a set of 113 questions that both humans and bots have answered.

That analysis follows the methodology we laid out before the [...]

---

Outline:

(02:00) Selecting a Bot Team

(03:16) Comparing the Bots to Pros

(06:33) Calibration Curves

(08:31) Discrimination Plots

(10:10) Scope Sensitivity

(11:45) Comparing Individual Bots to the Pros

(12:20) Metaculus Bots

(14:05) Discussion

The original text contained 9 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.