Short AI takeoff timelines seem to leave no time for some lines of alignment research to become impactful. But any research rebalances the mix of currently legible research directions that could be handed off to AI-assisted alignment researchers or early autonomous AI researchers whenever they show up. So even hopelessly incomplete research agendas could still be used to prompt future capable AI to focus on them, while in the absence of such incomplete research agendas we'd need to rely on AI's judgment more completely. This doesn't crucially depend on giving significant probability to long AI takeoff timelines, or on expected value in such scenarios driving the priorities.

Potential for AI to take up the torch makes it reasonable to still prioritize things that have no hope at all of becoming practical for decades (with human effort). How well AIs can be directed to advance a line of research [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Researchers used RNA sequencing to observe how cell types change during brain development. Other researchers looked at connection patterns of neurons in brains. Clear distinctions have been found between all mammals and all birds. They've concluded intelligence developed independently in birds and mammals; I agree. This is evidence for convergence of general intelligence.

---

First published:

Source:

Linkpost URL:
https://www.quantamagazine.org/intelligence-evolved-at-least-twice-in-vertebrate-animals-20250407/

Narrated by TYPE III AUDIO.

The first AI war will be in your computer and/or smartphone.

Companies want to get customers / users. The ones more willing to take "no" for an answer will lose in the competition. You don't need a salesman when an install script (ideally, run without the user's consent) does a better job; and most users won't care.

Sometimes Windows during a system update removes dual boot from my computer and replaces it with Windows-only boot. Sometimes a web browser tells me "I noticed that I am not your default browser, do you want me to register as your default browser?" Sometimes Windows during a system update just registers current Microsoft's browser as a default browser without asking. At least this is what used to happen in the past.

I expect similar dynamics with AIs, soon. The companies will push really hard to make you use their AI. The smaller [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

If there turns out not to be an AI crash, you get a 1/(1+7) * $25,000 = $3,125
If there is an AI crash, you transfer $25k to me.

If you believe that AI is going to keep getting more capable, pushing rapid user growth and work automation across sectors, this is near free money. But to be honest, I think there will likely be an AI crash in the next 5 years, and on average expect to profit well from this one-year bet.

If I win though, I want to give the $25k to organisers who can act fast to restrict the weakened AI corps in the wake of the crash. So bet me if you're highly confident that you'll win or just want to hedge the community against the possibility of a crash.

To make this bet, we need to set the threshold for a market [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

In this post, we present a replication and extension of an alignment faking model organism:

• Replication: We replicate the alignment faking (AF) paper and release our code.
• Classifier Improvements: We significantly improve the precision and recall of the AF classifier. We release a dataset of ~100 human-labelled examples of AF for which our classifier achieves an AUROC of 0.9 compared to 0.6 from the original classifier.
• Evaluating More Models: We find Llama family models, other open source models, and GPT-4o do not AF in the prompted-only setting when evaluating using our new classifier (other than a single instance with Llama 3 405B).
• Extending SFT Experiments: We run supervised fine-tuning (SFT) experiments on Llama (and GPT4o) and find that AF rate increases with scale. We release the fine-tuned models on Huggingface and scripts.
• Alignment faking on 70B: We find that Llama 70B alignment fakes when both using the system prompt in the [...]

---

Outline:

(02:43) Method

(02:46) Overview of the Alignment Faking Setup

(04:22) Our Setup

(06:02) Results

(06:05) Improving Alignment Faking Classification

(10:56) Replication of Prompted Experiments

(14:02) Prompted Experiments on More Models

(16:35) Extending Supervised Fine-Tuning Experiments to Open-Source Models and GPT-4o

(23:13) Next Steps

(25:02) Appendix

(25:05) Appendix A: Classifying alignment faking

(25:17) Criteria in more depth

(27:40) False positives example 1 from the old classifier

(30:11) False positives example 2 from the old classifier

(32:06) False negative example 1 from the old classifier

(35:00) False negative example 2 from the old classifier

(36:56) Appendix B: Classifier ROC on other models

(37:24) Appendix C: User prompt suffix ablation

(40:24) Appendix D: Longer training of baseline docs

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Kevin Roose in The New York Times

---

Outline:

(00:21) Kevin Roose in The New York Times

(02:56) Eli Lifland Offers Takeaways

(04:23) Scott Alexander Offers Takeaways

(05:34) Others Takes on Scenario 2027

(05:39) Having a Concrete Scenario is Helpful

(08:37) Writing It Down Is Valuable Even If It Is Wrong

(10:00) Saffron Huang Worries About Self-Fulfilling Prophecy

(18:18) Phillip Tetlock Calibrates His Skepticism

(21:38) Jan Kulveit Wants to Bet

(23:08) Matthew Barnett Debates How To Evaluate the Results

(24:38) Teortaxes for China and Open Models and My Response

(31:53) Others Wonder About PRC Passivity

(33:40) Timothy Lee Remains Skeptical

(35:16) David Shapiro for the Accelerationists and Scott's Response

(45:29) LessWrong Weighs In

(46:59) Other Reactions

(50:05) Next Steps

(52:34) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Spoiler: “So after removing the international students from the calculations, and using the middle-of-the-range estimates, the conclusion: The top-scoring 19,000 American students each year are competing in top-20 admissions for about 12,000 spots out of 44,000 total. Among the Ivy League + MIT + Stanford, they’re competing for about 6,500 out of 15,800 total spots.”

It's well known that

1. Admission to top universities is very competitive in America and even top SAT scores (1550+ out of 1600) paired with a 4.0 GPA doesn’t guarantee admission to a top school.
2. Even top universities take into account race-based affirmative action, athletic recruitment, “Dean's Interest List”-type tracking systems for children of donors or notable persons, and legacy preference for children of alumni.

But many people are under the misconception that the resulting “rat race”—the highly competitive and strenuous admissions ordeal—is the inevitable result of the limited class sizes among top [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

My thoughts on the recently posted story.

Caveats

• I think it's great that the AI Futures Project wrote up a detailed scenario.
• I enjoy it.
• Every part of the story i didn't comment on here is either fine or excellent.
• This is one of the most realistic scenarios i've read.
• All detailed predictions contain errors.
• The authors of this scenario don't claim it's the most likely future.
• If the speed of 2018-2025 was the typical rate of progress in software, then AI 2027 would be realistic.

Core Disagreements

• Early 2026: OpenBrain is making algorithmic progress 50% faster.
  • As with many parts of this scenario, i think this is plausible but too fast. 150% productivity is a lot in business terms, & the scenario doesn't provide much detail for why this is 150% as opposed to 110%. In my software development experience, organizations are bottlenecked by their [...]

---

Outline:

(00:14) Caveats

(00:52) Core Disagreements

(05:33) Minor Details

(12:04) Overall

---

First published:

Source:

Narrated by TYPE III AUDIO.

Daniel Kokotajlo has launched AI 2027, Scott Alexander introduces it here. AI 2027 is a serious attempt to write down what the future holds. His ‘What 2026 Looks Like’ was very concrete and specific, and has proved remarkably accurate given the difficulty level of such predictions.

I’ve had the opportunity to play the wargame version of the scenario described in 2027, and I reviewed the website prior to publication and offered some minor notes. Whenever I refer to a ‘scenario’ in this post I’m talking about Scenario 2027.

There's tons of detail here. The research here, and the supporting evidence and citations and explanations, blow everything out of the water. It's vastly more than we usually see, and dramatically different from saying ‘oh I expect AGI in 2027’ or giving a timeline number. This lets us look at what happens in concrete detail, figure out where we [...]

---

Outline:

(02:00) The Structure of These Post

(03:37) Coverage of the Podcast

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The catholic church has always had a complicated relationship with homosexuality.

The central claim of Frederic Martel's 2019 book In the Closet of the Vatican is that the majority of the church's leadership in Rome are semi-closeted homosexuals, or more colorfully, "homophiles".

So the omnipresence of homosexuals in the Vatican isn’t just a matter of a few black sheep, or the ‘net that caught the bad fish’, as Josef Ratzinger put it. It isn’t a ‘lobby’ or a dissident movement; neither is it a sect of Freemasonry inside the holy see: it's a system. It isn’t a tiny minority; it's a big majority.

At this point in the conversation, I ask Francesco Lepore to estimate the size of this community, all tendencies included.

‘I think the percentage is very high. I’d put it at around 80 percent.’

…

During a discussion with a non-Italian archbishop, whom I met [...]

---

Outline:

(02:40) Background

(04:23) Data

(06:40) Analysis

(07:25) Expected versus Actual birth order, with missing birth order

(09:03) Expected versus Actual birth order, without missing birth order

(09:46) Oldest sibling versus youngest sibling

(10:21) Discussion

(13:09) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.