Into AI Safety

Etienne Brisson and Allan Brooks, founder and program director of The Human Line Project, sat down with me to talk about AI-induced psychological harm, starting with their own experiences. Allan spent a month spiraling with ChatGPT, ending with him warning national security agencies about an encryption break that never happened. Etienne's uncle became convinced his chatbot was sentient and was hospitalized still messaging it from the back of a police car.

From there we get into why the community rejects "AI psychosis," what's driving these spirals, and whether anthropomorphic language is doing more damage than we admit. We also get into Allan's litigation against OpenAI, why Etienne thinks consequences are the only substitute for missing incentives, and whether these concerns are just a new moral panic. If you're interested in listening in greater detail, check out the extended version on the Kairos.fm Patreon.

The Human Line Project runs a free peer support community. If you're struggling, findahelpline.com will route you to support wherever you are.

NOTE: Jacob volunteers some of his time with The Human Line Project.

Chapters

• (00:00) - Intro

• The Human Line Project website
• The Human Line community
• The Logic article - Meet the Quebecer Leading the Human Resistance Against AI
• NY Times article - Chatbots Can Go Into a Delusional Spiral. Here's How It Happens.
• Clear-Eyed AI blogpost - Practical Tips for Reducing Chatbot Psychosis
• Georgetown Institute for Technology Law & Policy brief - AI Sycophancy & OpenAI
• FAccT '26 paper - Characterizing Delusional Spirals through Human-LLM Chat Logs
• Nature op-ed - Friendlier LLMs Tell Users What They Want to Hear — Even When It Is Wrong
• MMLC Group blogpost - China Releases Rules for AI "Human-Like" Emotional Interaction Services
• Into AI Safety podcast episode w/ Jacy Reese-Anthis (includes a section on ELIZA)

Dr. Evan Miyazono joins Jacob to talk about his path from quantum networking and Protocol Labs to founding Atlas Computing. They discuss why Atlas moved away from formal methods research and toward what Evan calls “field strategy”: identifying important problems, figuring out what interventions could actually address them, and finding the people best positioned to make them happen. The conversation also covers the resilience gap map, coordination failures, AI safety, "value alignment," and the question of how to decide what work is worth doing when there’s no obvious organization responsible for it.

Check out the video version of this podcast on the Kairos.fm YouTube channel, or the extended version on Patreon!

Chapters

• (00:00) - Intro

• Atlas Computing website
• The AI Resilience Gap Map
• Atlas Computing blogpost - Civilization's Maintenance Backlog
• Atlas Computing blogpost - Your Solution Doesn't Know Your Problem Exists
• Galois blogpost - What Are Formal Methods?
• Patrick Collison & Tyler Cowen essay discussion - We Need a New Science of Progress
• Federation of American Scientists article - Focused Research Organizations

What if the safest AI models weren't built by adding guardrails after training, but by shaping what gets learned in the first place? Ethan Roland, senior alignment researcher at AE Studio and first author on an ICML 2026 spotlight paper, joins Jacob to talk about gradient routing, a technique that routes dangerous capabilities into isolated parts of a model's architecture where they can be locked or removed entirely. They get into the absorption effect, KYC-style access control frameworks, and what it would actually take for frontier labs to adopt this kind of work before it's needed rather than after.

Chapters

• (00:00) - Introduction

• Ethan's website
• The paper landing page
• Preprint - Gradient Routing: Masking Gradients to Localize Computation in Neural Networks
• ICLR paper and webpage - Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
• Wikipedia article - CBRN defense
• NTI tutorials on bioweapons and nuclear testing

Today's episode does double duty as an interview and an announcement. Joe Williams, host of the new Kairos.fm show "Beyond the Paycheck: Reclaiming the Case for UBI in the Age of AI," joins Jacob to talk about his background as a freelance translator and how AI quietly dismantled his livelihood in 2025. From there the conversation expands into whether this moment is really different from past waves of automation, who exactly makes up the tech billionaire class, and why you should probably raise an eyebrow when someone like Elon Musk says he supports UBI. Go check out Reclaiming UBI!

Chapters

• (00:00) - Intro

• NEW KAIROS.FM PODCAST - Reclaiming UBI: Work and Values in the Age of AI
• Josh Steimle blog post - Will AI Destroy Jobs? 500 Years of Predictions Say No
• Smithsonian Magazine article - What the Luddites Really Fought Against
• ITIF report - Oops: The Predicted 47 Percent of Job Loss From AI Didn't Happen
• Frontiers in Artificial Intelligence journal article - AI, Universal Basic Income, and Power: Symbolic Violence in the Tech Elite's Narrative
• Time article - What to Know About Worldcoin and the Controversy Around It
• Wikipedia page - World (blockchain)

Zainab Majid, co-founder of Asymmetric Security, joins Jacob for a conversation on the intersection between AI Safety and cybersecurity, as well as the future of digital forensics. Drawing from years of incident response work, she explains how cyber attacks actually unfold, why AI is changing both offense and defense, and how her team is building AI-native tools to investigate breaches faster and more effectively. Other topics explored in this episode include trust in the AI/cybersecurity industries, the realities behind cybersecurity hype, and the challenge of keeping humans meaningfully involved as these systems become more capable. Zainab also gives practical, actionable advice on how you can protect yourself.

If you're interested in over 30 minutes of additional content, head on over to the Kairos.fm Patreon where you can become a subscriber for just $2 per month, which helps make this whole podcasting thing a bit more sustainable.

Chapters

• (00:00) - Intro

• Asymmetric Security website
• EvalEval @ NeurIPS workshop paper - Rethinking CyberSecEval
• Meta AI report - Purple Llama CyberSecEval
• Anthropic press release - Project Glasswing
• Schneier on Security blogpost - What Anthropic’s Mythos Means for the Future of Cybersecurity

Technology has been moving faster than policy for some time now, and the advent of AI isn't changing that, so what can we do to maintain safety despite uncertainty? Su Cizem has spent the last few years trying to answer that question. As an analyst at the Future Society, she works on global AI governance, specifically on building international consensus around AI red lines: the thresholds we collectively agree must never be crossed. In this conversation, Su walks through her path from philosophy to policy, the evolution of the global AI safety summit series, why voluntary commitments from AI labs aren't enough, and what it would actually take to make international cooperation on AI safety real.

Chapters

• (00:00) - Introduction

• Su's LinkedIn
• Global Call for AI Red Lines
• The Futures Society report - “Facing the Stakes of AI Together”: 2025 Athens Roundtable Report
• Politico article - How the global effort to keep AI safe went off the rails
• TechPolicy.Press article - A Timeline of the Anthropic-Pentagon Dispute
• The Guardian article - AI got the blame for the Iran school bombing. The truth is far more worrying
• Google and OpenAI Employee open letter - We Will Not Be Divided
• The Register article - Altman said no to military AI abuses – then signed Pentagon deal anyway
• SaferAI report - Evaluating AI Providers’ Frontier AI Safety Frameworks

Jacy Reese-Anthis, founder of Sentience Institute and researcher at Stanford, began his journey working for animal welfare, but is now finishing up his PhD with research in many different AI subfields at the intersection of neuroscience, philosophy, social science, and machine learning. While this may seem like an odd jump at first, Jacy shares how his work has all been centered around the idea of moral circle expansion. In this episode, we dig into what sentience actually means (or at least how we can begin to think about it), why anthropomorphization is more complicated than it sounds, and how language models may be able to be leveraged as an effective tool for social science research.

Jacy also shares his median AGI estimate somewhere in there, so stay tuned if you want to catch it.

As part of my effort to make this whole podcasting thing more sustainable, I have created a Kairos.fm Patreon which includes an extended version of this episode. Supporting gets you access to these extended cuts, as well as other perks in development.

Chapters

• (00:00) - Introduction

• Jacy's website
• Wikipedia article - Jacy Reese Anthis
• Sentience Institute website
• CHI paper - Digital Companionship: Overlapping Uses of AI Companions and AI Assistants
• ICML paper - LLM Social Simulations Are a Promising Research Method
• ACL paper - The Impossibility of Fair LLMs
• Wikipedia article - ELIZA effect
• The Atlantic article - How a Google Employee Fell for the Eliza Effect

What does it actually take to build a successful AI safety organization? I'm joined by Dr. Ryan Kidd, who has co-led MATS from a small pilot program to one of the field's premier talent pipelines. In this episode, he reveals the low-hanging fruit in AI safety field-building that most people are missing: the amplifier archetype.

I pushed Ryan on some hard questions, from balancing funder priorities and research independence, to building a robust selection process for both mentors and participants. Whether you're considering a career pivot into AI safety or already working in the field, this conversation offers practical advice on how to actually make an impact.

Chapters

• (00:00) - - Intro

• MATS

Ryan's Writing

• LessWrong post - Talent needs of technical AI safety teams
• LessWrong post - AI safety undervalues founders
• LessWrong comment - Comment permalink with 2025 MATS program details
• LessWrong post - Talk: AI Safety Fieldbuilding at MATS
• LessWrong post - MATS Mentor Selection
• LessWrong post - Why I funded PIBBSS
• EA Forum post - How MATS addresses mass movement building concerns

FTX Funding of AI Safety

• LessWrong blogpost - An Overview of the AI Safety Funding Situation
• Fortune article - Why Sam Bankman-Fried’s FTX debacle is roiling A.I. research
• NY Times article - FTX probes $6.5M in payments to AI safety group amid clawback crusade
• Cointelegraph article - FTX probes $6.5M in payments to AI safety group amid clawback crusade
• FTX Future Fund article - Future Fund June 2022 Update (archive)
• Tracxn page - Anthropic Funding and Investors

Training & Support Programs

• Catalyze Impact
• Seldon Lab
• SPAR
• BlueDot Impact
• YCombinator
• Pivotal
• Athena
• Astra Fellowship
• Horizon Fellowship
• BASE Fellowship
• LASR Labs
• Entrepeneur First

Funding Organizations

• Coefficient Giving (previously Open Philanthropy)
• LTFF
• Longview Philanthropy
• Renaissance Philanthropy

Coworking Spaces

• LISA
• Mox
• Lighthaven
• FAR Labs
• Constellation
• Collider
• NET Office
• BAISH

Research Organizations & Startups

• Atla AI
• Apollo Research
• Timaeus
• RAND CAST
• CHAI

Other Sources

• AXRP website - The AI X-risk Research Podcast
• LessWrong blogpost - Shard Theory: An Overview

Sean McGregor and I discuss about why evaluating AI systems has become so difficult; we cover everything from the breakdown of benchmarking, how incentives shape safety work, and what approaches like BenchRisk (his recent paper at NeurIPS) and AI auditing aim to fix as systems move into the real world. We also talk about his history and journey in AI safety, including his PhD on ML for public policy, how he started the AI Incident Database, and what he's working on now: AVERI, a non-profit for frontier model auditing.

Chapters

• (00:00) - Intro

• Sean McGregor's website
• AVERI website

BenchRisk

• BenchRisk website
• NeurIPS paper - Risk Management for Mitigating Benchmark Failure Modes: BenchRisk
• NeurIPS paper - AI and the Everything in the Whole Wide World Benchmark

AIID

• AI Incident Database website
• IAAI paper - Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database
• Preprint - Lessons for Editors of AI Incidents from the AI Incident Database
• AIAAIC website (another incident tracker)

Hot AI Summer

• CACM article - A Few Useful Things to Know About Machine Learning
• CACM article - How the AI Boom Went Bust
• Undergraduate Thesis - Analyzing the Prospect of an Approaching AI Winter
• Tech Genies article - AI History: The First Summer and Winter of AI
• CACM article - There Was No ‘First AI Winter’

Measuring Generalization

• Neural Computation article - The Lack of A Priori Distinctions Between Learning Algorithms
• ICLR paper - Understanding deep learning requires rethinking generalization
• ICML paper - Model-agnostic Measure of Generalization Difficulty
• Radiology Artificial Intelligence article - Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls
• Preprint - Quantifying Generalization Complexity for Large Language Models

Insurers Exclude AI

• Financial Times article - Insurers retreat from AI cover as risk of multibillion-dollar claims mount
• Tom's Hardware article - Major insurers move to avoid liability for AI lawsuits as multi-billion dollar risks emerge — Recent public incidents have lead to costly repercussions
• Insurance Newsnet article - Insurers Scale Back AI Coverage Amid Fears of Billion-Dollar Claims
• Insurance Business article - Insurance’s gen AI reckoning has come

Section 230

• Section 230 overview
• Legal sidebar - Section 230 Immunity and Generative Artificial Intelligence
• Bad Internet Bills website
• TechDirt article - Section 230 Faces Repeal. Support The Coverage That’s Been Getting It Right All Along.
• Privacy Guides video - Dissecting Bad Internet Bills with Taylor Lorenz: KOSA, SCREEN Act, Section 230
• Journal of Technology in Behavioral Health article - Social Media and Mental Health: Benefits, Risks, and Opportunities for Research and Practice
• Time article - Lawmakers Unveil New Bills to Curb Big Tech’s Power and Profit
• House Hearing transcript - Legislative Solutions to Protect Children and Teens Online

Relevant Kairos.fm Episodes

• Into AI Safety episode - Growing BlueDot's Impact w/ Li-Lian Ang
• muckrAIkers episode - NeurIPS 2024 Wrapped 🌯

Other Links

• Encyclopedia of Life website
• IBM Watson AI XPRIZE website
• ML Commons website
• Wikipedia article

Philosopher Dr. David Thorstad tears into one of AI safety's most influential arguments: the singularity hypothesis. We discuss why the idea of recursive self-improvement leading to superintelligence doesn't hold up under scrutiny, how these arguments have redirected hundreds of millions in funding away from proven interventions, and why people keep backpedaling to weaker versions when challenged.

David walks through the actual structure of singularity arguments, explains why similar patterns show up in other longtermist claims, and makes the case for why we should focus on concrete problems happening right now like poverty, disease, the rise of authoritarianism instead of speculative far-future scenarios.

Chapters

• (00:00) - Intro

• David's personal website
• Reflective Altruism, David's blog

The Singularity Hypothesis

• David's Philosophical Studies article - Against the singularity hypothesis
• Time "AI Dictionary" page - Singularity
• EA Forum blogpost - Summary: Against the singularity hypothesis
• Journal of Conciousness Studies article - The Singularity: A Philisophical Analysis
• Interim Report from the Panel Chairs: AAAI Presidential Panel on Long-Term AI Futures
• Epoch AI blogpost - Do the returns to software R&D point towards a singularity?
• Epoch AI report - Estimating Idea Production: A Methodological Survey

Funding References

• LessWrong blogpost - An Overview of the AI Safety Funding Situation
• AISafety.com funding page
• Report - Stanford AI Index 2025, Chapter 4.3
• Forbes article - AI Spending To Exceed A Quarter Trillion Next Year
• AI Panic article - The “AI Existential Risk” Industrial Complex
• GiveWell webpage - How Much Does It Cost To Save a Life?
• Wikipedia article - Purchasing power parity

Pascal's Mugging and the St. Petersburg Paradox

• Wikipedia article - St. Petersburg Paradox
• Conjecture Magazine article - Pascal’s Mugging and Bad Explanations
• neurabites explainer - Ergodicity: the Most Over-Looked Assumption
• Wikipedia article - Extraordinary claims require extraordinary evidence

The Time of Perils

• Global Priorities Institute working paper - Existential risk pessimism and the time of perils
• Ethics article - Mistakes in the Moral Mathematics of Existential Risk
• Philosophy & Public Affairs article - High Risk, Low Reward: A Challenge to the Astronomical Value of Existential Risk Mitigation
• Toby Ord Book - The Precipice
• Rethink Priorities blogpost - Charting the precipice
• AI Futures Project blogpost - AI 2027

Trump's Higher Education Threat Compact

• Wikipedia article - Compact for Academic Excellence in Higher Education
• Pen America explainer - What is Trump’s Compact for Higher Education? And More Frequently Asked Questions
• Statement by the Vanderbilt AAUP Executive Committee on the “Compact for Academic Excellence in Higher Education”
• The Vanderbilt Hustler article - BREAKING: Chancellor Daniel Diermeier fails to reject higher education compact, reaffirms Vanderbilt’s values and openness to discussion
• The Vanderbilt Hustler article - Students and faculty organize rally outside Kirkland Hall against Trump administration’s higher education compact
• Free Speech Center article - Compact for Academic Excellence

More of David's Work

• Global Priorities Institute working paper - What power-seeking theorems do not show
• Book - Essays on Longtermism

Vibe Shift

• Blood in the Machine article - GPT-5 Is a Joke. Will It Matter?
• Futurism article - Evidence Grows That GPT-5 Is a Bit of a Dud
• Gary Marcus substack - GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
• Pew Research report - How the U.S. Public and AI Experts View Artificial Intelligence
• N...