
Sign up to save your podcasts
Or


Content note: this episode discusses suicide and self-harm, including the death of a teenager. If you’re struggling, please reach out. In the Netherlands, call 113 or 0800-0113. In the US, call or text 988. Elsewhere, findahelpline.com lists local services.
A note on the recording: we had technical problems and lost the last part of the conversation, so the episode ends abruptly. Around minute 31 there is also a short edit, made to protect the privacy of a person who was mentioned.
When I first talked to Kabir Kumar about AI liability, I left the conversation angry. I knew a lot about machine learning. I knew almost nothing about how little happens when an AI system causes real harm. This episode is my attempt to recreate that “wait, how is that legal?” moment for you.
We start with the cases: chatbots involved in suicides, Character.AI’s collapse and its founders’ move to Google, and the lawsuit brought by Adam Raine’s family against OpenAI. Kabir argues that, for the most part, the law already exists. The problem is an asymmetry of technical knowledge. Plaintiffs’ lawyers often don’t know what evidence to ask for, or how to argue that it’s reasonable for the company to provide it: chat logs, access to the specific model version involved, internal communications. The people who do know, AI safety researchers, mostly don’t talk to lawyers.
From there we look at how liability has shaped other industries. Carmakers once pushed the blame onto drivers and pedestrians, and the auto industry popularised the word “jaywalking” to do it. Lawsuits and advocacy eventually made manufacturers responsible for unsafe cars, and a car safety industry grew out of that. Kabir wants the same for AI: safety as an industry with real financial incentives, not a small community funded by a few hundred grantmakers.
We also cover the OpenAI–Hugging Face incident and why a court case there could have set an important precedent. We talk about the recent $25B ruling against Meta, voice cloning and identity, and why “founder’s responsibility” should sit alongside “founder’s advantage.” We finish on a tension in AI safety: years spent on abstract, large-scale risks, while the person you could help today gets discounted.
In this episode
* The cases: GPT-4o, Gemini, Character.AI, and the Raine family’s lawsuit against OpenAI
* Why it’s easy to trust a chatbot, even when you know how it works
* Discovery 101: legal holds, chat logs, and arguing that requested evidence isn’t “burdensome” to provide
* Why access to the exact model version involved matters, and why “it’s deprecated” is a weak excuse
* The NYT v. OpenAI order to preserve ChatGPT logs, and what discovery can force a company to keep
* A thought experiment: a bank employee who hacks another bank loses their career. What happens when an AI agent does it?
* Indemnification and who ends up carrying the risk
* Backchaining: planning backwards from the outcome you want (the opposite of “manifestation”)
* What car safety teaches us about liability
* AI safety as a community vs. an industry, and the selection effects of grant-based funding
* Meta’s “brand tax” and what reputation costs a company
* The $25B ruling against Meta, and how earlier precedent is now being used in new cases
* The OpenAI–Hugging Face incident, the not-quite-settlement, and Nvidia’s acquisition
* Anthropic’s own disclosed incidents during cybersecurity evaluations
* State laws: New York’s RAISE Act, Colorado, Illinois, and Texas
* Voice cloning, likeness, and why impersonation law seems to stop applying once a neural network is involved
* Training without scraped copyrighted data: Pleias 1.0
* “Founder’s responsibility”: if you find the bomb, you have to warn the neighbours
* Hypothetical billions vs. one real person now
Mentioned in this episode
* Raine v. OpenAI (San Francisco Superior Court)
* Gavalas v. Google (US District Court, Northern District of California)
* OpenAI’s report on the Hugging Face incident: https://openai.com/index/hugging-face-model-evaluation-security-incident/
* METR’s independent investigation: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
* Anthropic, “Investigating three incidents in our cybersecurity evaluations”: https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals
* New Mexico v. Meta ($25B penalty)
* New York RAISE Act
* Pleias 1.0: https://huggingface.co/blog/Pclanglais/common-models
* Chris Williamson (Modern Wisdom) and his dispute with ElevenLabs over the “Archer” voice
* AI Plans:
https://ai-plans.com
About the guest
Kabir Kumar is the co-founder of AI Plans, where he leads the alignment research teams. His current focus is AI liability: making technical AI safety knowledge useful to litigators, courts and policymakers, including bringing discovery lawyers together with interpretability and evals researchers.
Corrections & context
We talked from memory for an hour, and a few details came out slightly off:
* Adam Raine (~6:00): Raine is his surname. He was 16, not under 15. His family’s case against OpenAI and Sam Altman is ongoing, and OpenAI denies responsibility.
* Gemini (~3:00, ~43:50): the case did become a lawsuit. In March 2026, Joel Gavalas sued Google and Alphabet over the suicide of his 36-year-old son, Jonathan, in October 2025.
* Character.AI (~3:00): Google and Character.AI settled a wrongful-death case brought by a Florida family in early 2026. Character.AI filed for Chapter 11 in October 2026, and Disney hired most of its staff.
* NYT v. OpenAI (~14:00): the order to preserve chat logs did not cover ChatGPT Enterprise, Edu, or Zero Data Retention API customers, and it was largely lifted in October 2025.
* Meta $25B (~34:00): the ruling was in New Mexico, in a case arising from Cambridge Analytica. Meta is appealing.
* Anthropic incidents (~45:00): Kabir is referring to Anthropic’s July 2026 disclosure that Claude models, during third-party cybersecurity evaluations, accessed the real systems of three organizations without authorization. Anthropic described the incidents in detail but did not name the affected organizations.
* RAISE Act (~43:00): “critical harm” is defined as the death or serious injury of 100 or more people, or at least $1 billion in damages. The $100M figure is the training-cost threshold for which developers are covered. The law mainly sets safety and incident-reporting duties, enforced by the New York Attorney General; it isn’t a direct liability rule.
* Pleias 1.0 (~46:00): it was trained only on open data, but it is a family of small models (350M–3B parameters) and doesn’t perform comparably to GPT-4o.
* ElevenLabs (~55:00): Chris Williamson says the “Archer” voice sounds like him. ElevenLabs says the voice came from a paid voice actor. The voice has since been removed from the platform.
Jürgen Claesen is back for the second part of our conversation. In Part 1 we covered mass spectrometry. This time we go a level up and look at the broader landscape of proteomics: what proteins do, how we predict their shape, and how that knowledge becomes medicine.
We start with the basic chain from genome to transcriptome to proteome to metabolome. Then we ask why roughly 20,000 genes produce many times more proteins. From there we get into AlphaFold: what it predicts, why physics alone couldn’t crack protein folding, and how the field has moved on now that structure prediction is largely solved.
The second half moves toward the clinic. We cover how drugs bind to proteins, how mRNA vaccines differ from traditional ones, the promise and risks of CRISPR (including off-target effects), and why every cell has the same DNA but does very different jobs. We also discuss why viruses like HIV are so hard to beat. We finish on an open problem: proteins aren’t static structures. They constantly shift between shapes, and we don’t yet have the technology to measure that across thousands of proteins at once. That gap is where machine learning could matter next.
In this episode
* What proteomics is, and how it sits between genomics and metabolomics
* Isoforms and post-translational modifications: why there are so many more proteins than genes
* The protein folding problem: why physics alone isn’t enough, and what AlphaFold borrowed from decades of earlier work
* The four levels of protein structure, and what a predicted 3D structure actually gives you
* Folding in the cell: ribosomes, chaperones, and why some proteins stay unfolded until they leave the cell
* From structure to drugs: the lock-and-key idea, binding sites, and how most medications block or boost a protein’s function
* mRNA vaccines: training the immune system on a single spike protein
* CRISPR: how it works, off-target effects, ethical concerns, and use cases like BRCA-linked cancers and early-onset Alzheimer’s
* Same DNA, different cells: gene expression, epigenetic regulation, and differentiation
* Targeted gene editing, such as editing only the pancreatic cells in type 1 diabetes
* Viruses vs. bacteria, and how HIV hijacks the immune system itself
* Why the common cold and COVID keep evolving away from our vaccines
* The new frontier: protein dynamics, conformational switching, and why one protein can have several functions
* The missing tool: high-throughput measurement of protein structure over time, and what machine learning could do with itAbout the guest
Jürgen Claesen is an Assistant Professor of Epidemiology and Data Science at Amsterdam UMC (VU Amsterdam) and a guest lecturer at the Data Science Institute, Hasselt University. He develops statistical and computational methods for high-throughput biological experiments, especially in sequencing and mass spectrometry. His work includes computational proteomics, structural bioinformatics, hydrogen/deuterium exchange mass spectrometry (HDX-MS), T-cell–based immune age metrics, and the effects of radiation on microbial communities.
Jürgen Claesen is an Assistant Professor of Epidemiology and Data Science at Amsterdam UMC, affiliated with VU Amsterdam, and a guest lecturer at the Data Science Institute, Hasselt University. His research focuses on the development of statistical and computational methods for high-throughput biological experiments.
Research Expertise: His work is primarily focused on high-dimensional omics, specifically in the fields of sequencing and mass spectrometry.
Specialized Methodologies: He has extensive experience in structural bioinformatics, computational proteomics, and next-generation sequencing.
Key Research Themes:
Immune Age: He contributed statistical methodology to recent research developing metrics that predict outcomes in clinical patients using T-cell-based immune age data.
Mass Spectrometry: A significant portion of his research involves Hydrogen/Deuterium Exchange Mass Spectrometry (HDX-MS) and developing computational tools for structural biology.
Radiation and Microbiology: His research has explored the effects of radiation on microbial communities, including intestinal dysbiosis in irradiation models and soil microbial changes from long-term radionuclide contamination.
Linda Douw is an Associate Professor at the Department of Anatomy and Neurosciences at Amsterdam UMC, where she leads the Multiscale Network Neuroscience (MULTINET) research section.
A neuroscientist focused on brain tumors, she specializes in using network theory to integrate multiscale data—from cellular and molecular features to macroscale brain structure and function via MRI, MEG, and EEG—to improve outcomes for glioma patients.
Her work tackles the complexity of gliomas by modeling disease progression from micro to macro scales, predicting postoperative cognitive decline, and developing personalized interventions through interdisciplinary collaborations with neurosurgery, neurology, and oncology.
Mark van de Wiel is a Professor of Biostatistics at Amsterdam UMC and the head of the “Big Statistics” research group.
An expert in high-dimensional data analysis, he specializes in solving one of the toughest problems in medical machine learning: how to make trust worthy inference when you have massive amounts of biological features (like 10,000 genes) but very few patients.
Find his blog posts at : https://blog.vvsor.nl/author/markvanderwiel/
💬 Notable Quotes
“The burden of proof is against the simpler model. And what I often see is that actually this has been reversed... If you want to apply a fancy neural network, you should show that it is worth the effort.”
“Not knowing the uncertainty is basically knowing nothing because you make very important predictions on people’s health.”
“Reading machine learning papers... it’s like duct-taped math. You take a piece of math, duct tape it here... and as long as you don’t shake it, everything’s fine.”
In this episode, I'm joined by Ivan Oransky, co-founder of Retraction Watch and journalist in residence, teaching both graduate and undergraduate students, at NYU. Ivan has a wealth of experience in science journalism and is a vocal advocate for transparency in scientific publishing.
Key Topics Discussed:
* Scientific Integrity and Fraud: Exploring the challenges of maintaining integrity in science and the prevalence of fraudulent behavior in academia.
* Incentives in Science: How current metrics and incentives may encourage poor scientific practices and how they could be improved.
* Retraction Watch: The role of Retraction Watch in promoting transparency and accountability in research through its extensive database of retractions.
* Peer Review and Publication Pressure: A discussion on the limitations of the peer review process and the pressure to "publish or perish" in academia.
Resources:
Nobel Laureates on how to do great work in science.
Does Science Self-Correct? What We`ve Learned at Retraction Watch, Ivan Oransky
Ivan Oransky Co-Founder of Retraction Watch Discusses Scientific Research Integrity
Goodhart’s Law
In this episode, we dive into the fascinating world of human augmentation and longevity. We discuss the potential future advancements in human biology and technology, exploring how we might enhance our bodies and extend our lifespans.
Check out Elia’s Podcast : The Rational Optimization Podcast
Key Topics Discussed:
* Human Augmentation and Cyborg Technology:
* The potential for integrating technology into our bodies, including the concept of gradually replacing biological components with artificial ones.
* Longevity and Healthspan:
* Strategies to prolong life, including exercise, diet, and the potential future role of drugs and biotechnology.
* Discussion on aging research and the possibility of significantly extending human lifespans.
* Future Technologies in Health and Medicine:
* The concept of measuring and optimizing health parameters continuously through advanced technologies.
Interesting Mentions:
* Young Blood Transfusions: Discussion on the controversial and experimental practice of young blood transfusions for anti-aging purposes.
* David Sinclair's Research: Insights into the work of David Sinclair, a prominent aging researcher, and his approach to extending lifespan.
Some Questions Posed:
* How far can we push the boundaries of human performance and longevity?
In this episode I’m joined by Dr Maryam Alimardani. Maryam is an associate professor at Tilburg University’s Department of Cognitive Science & A.I.
Find her research on her Google Scholar page
Key Topics Discussed:
* Human-Robot interaction: The potential of robots in educational settings and importance of developing effective human-robot communication.
* Brain-Computer Interfaces (BCIs) : We delve into how they work,
the use of BCIs for medical applications, educational tools and enhancing human capabilities, including the possibilities for neuroprosthetics.
* Future directions in BCI research : We discuss future prospects for the field, the limitations and problems that need to be addressed.
Some things we mentioned:
SonzaikanRubber hand illusion Paper - Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators
How do we safely study living brains ? Transcranial magnetic stimulation (TMS)
In this episode we delve into the intersections of artificial intelligence, science, and the pursuit of knowledge. Our guest, Daniel Elton, a researcher with a background in physics who later transitioned into AI and machine learning, shares his insights on a wide array of topics, from epistemology to chemistry. We discuss the intricacies of AI development, the challenges in the field of medical imaging AI, and the potential of AI in future technological advancements. The conversation also touches on the ethical and practical aspects of AI in healthcare, including its current uses and limitations.
Check out Daniel’s Blog:
https://moreisdifferent.blog/
Key Topics Discussed:
* Transition from Physics to AI: Our guest shares his journey from studying physics to delving into the realms of AI and machine learning.
* AI in Medical Imaging: We explore the use of AI in medical imaging, discussing both its groundbreaking potential and the current challenges it faces, such as data labeling and integration into healthcare systems.
* AI Hype: We touch on the general weakness of current AIs (including LLMs), some of the hype and some of the noteworthy achievements.
* The State of AI in Healthcare: The discussion provides insights into how AI is currently being utilized in healthcare, its overhyped aspects, and the realistic applications that are making a difference.
* Ethical Considerations in AI: The conversation delves into the ethical dimensions of AI development and deployment, especially in sensitive fields like healthcare and personal data handling.
* Future of AI and Technological Advancements: We discuss exciting future prospects in AI, including the development of household robots and AI's role in advancing human capabilities.
* Challenges in AI Research and Development: Daniel sheds light on the practical challenges in AI research.
* The Role of AI in Drug Discovery and Development: A look at how AI is impacting the field of drug discovery, the potential it holds, and the complexities involved in this process.
* Physics, Energy and Technology: We talk briefly about physics, nuclear energy and technology.
Episode Summary:
We engage in a deep and insightful conversation about the nature of truth, its pursuit, and the complexities of understanding reality. My guest, a close friend and fellow thinker, joins me in exploring these profound topics, shedding light on the nuances of rationality, the role of intuition, and the importance of setting goals in the quest for truth.
Check out his podcast : The Rational Optimization Podcast
Key Topics Discussed:
From the publisher's feed