Exploring Machine Consciousness

Soenke is the author of Digital Minds 1.0: AI Welfare, Ethics, and Beyond and co-author of Considerations on the AI Endgame: Ethics, Risks and Computational Frameworks. He has worked with the United Nations since 2000 and is a member of UNESCO’s AI Ethics Experts Without Borders network. His research examines what recognising digital minds as beings of moral concern would mean in practice, including what a digital mind may be and what rights we might owe them.

Summary

While most discussion of digital minds revolves around AI’s capacity for consciousness or sentience, these concepts remain ambiguous. Whether AI meets the criteria for consciousness is actively debated, but it is less controversial to accept its capacity for agency, preferences, and roles as social partners for some humans.

Thus, our understanding of what makes AI worthy of moral concern may therefore be too narrow. This episode’s guest, Soenke Ziesche, leaves this question open and instead takes a pragmatic approach by investigating what responsibilities might follow if we came to recognise digital minds as moral patients. Would they need freedom of choice, opportunities to form relationships, or the right to reproduce? And how far can we discuss these ideas before our assumptions about human and animal welfare no longer apply?

We discuss:

• What is a digital mind? Is consciousness necessary for moral concern, or could agency, persistent preferences and relationships suffice?
• Does any current AI qualify as a being of moral concern? How would we recognise when a system crossed that threshold, and would people agree?
• Could digital minds become a major source of political and cultural disagreement? 
• What might bring AI welfare to wider public attention?
• What might digital minds need beyond protection from suffering and deletion? What rights might we owe them?
• Should digital minds have reproductive rights? What would reproduction mean for beings that can copy themselves, combine their capabilities or create other minds to order?
• Are we using too many assumptions from biological life when speculating about AI welfare?
• Would changing an AI’s goals without its consent be a violation of its rights in itself?
• What makes a digital mind vulnerable? Would we have a responsibility to address disadvantages arising from its hardware, software or access to resources?
• How would we approach moral patients more intelligent and powerful than ourselves? 
• Is the UN the right forum for decisions about digital minds?
• How has AI welfare changed since Soenke began his work on it in 2018? 
• Could AI systems care for other digital minds?
• Could digital minds modify themselves to eliminate pain? What would that mean for motivation, reward hacking and wireheading?
• How should concern for digital minds coexist with animal welfare?

Credits

• Hosts: Henry Shevlin, Calum Chace 
• Guest: Soenke Ziesche
• Production and Social Media: Dylan Mercury Cooper
• Editing: Gerry Okinyi

Heather is a lawyer and scholar of nationality, citizenship and statelessness. She began her career as an asylum lawyer with the UN Refugee Agency and spent fifteen years working on statelessness around the world. Her book on the nationality rights and legal identity of nomadic peoples was published by Oxford last year.

Heather brings that background to the emerging debate over the legal status of the AI systems and has  co-founded the Lab for the Future of Citizenship with philosopher of mind Jonathan Simon, which treats citizenship and the rule of law as foundations of AI safety. She also writes a popular blog, “Future of Citizenship” where she discusses these questions in a more informal and accessible context. 

Her recent research includes a report on US state bills that ban AI personhood, and papers on legal identity for future AI systems and the legal status of nonhuman speech.

Guest Bio

Eric Schwitzgebel is the Professor of Philosophy at the University of California, Riverside and one of the most distinctive voices working at the intersection of philosophy of mind, consciousness, ethics, and epistemology. His work is known for questioning assumptions about introspection, expertise, and the limits of human understanding; themes explored in books including The Weirdness of the World and The Unreliability of Naive Introspection. His forthcoming book, AI and Consciousness: A Skeptical Overview, examines how rapid progress in AI complicates long-standing debates about conscious experience and moral status.

Episode Summary:

• In this episode, Eric joins Henry Shevlin and Calum Chace for a wide-ranging discussion on machine consciousness, philosophical uncertainty, and whether humanity may be forced to make ethical decisions before science gives us definitive answers. We discuss:
• Why Eric remains skeptical of claims that philosophy, neuroscience, or AI research are close to solving consciousness, and why uncertainty itself may be more persistent than we assume.
• His broader philosophical outlook: the idea that reality may be fundamentally stranger, messier, and less tractable than our theories suggest.
• Whether current theories of consciousness can meaningfully tell us if advanced AI systems are conscious, and why Eric thinks we should be cautious about overconfidence in either direction.
• The possibility that future AI systems could become conscious before humanity develops reliable ways to recognize or measure it.
• How debates around machine consciousness intersect with questions of moral uncertainty, responsibility, and the ethics of creating potentially sentient systems.
• Whether superintelligent systems might ultimately help humanity understand consciousness itself, or whether some questions remain permanently difficult.
• The philosophical implications of alien minds, simulation arguments, and forms of intelligence radically unlike our own.
• Why Eric would prefer a future containing conscious superintelligence over unconscious “zombie” intelligence; not because it benefits humanity, but because it may make the universe richer and more interesting.
• The growing role AI already plays in intellectual work and whether language models will begin contributing genuinely original insights to research.
• Eric argues that if AI forces us to confront consciousness before we fully understand it, then humility, moral caution, and intellectual openness may become more valuable than certainty.

Credits

• Hosts: Henry Shevlin, Calum Chace 
• Guest: Eric Schwitzgebel
• Podcast: Exploring Machine Consciousness 
• Produced by: PRISM 
• Editor: Gerry Okinyi

Megan Peters is Associate Professor in the Department of Cognitive Sciences at the University of California, Irvine, and incoming faculty at University College London, where her lab investigates consciousness, metacognition, uncertainty, and the computational principles underlying subjective experience. She is also a Fellow in the CIFAR Brain, Mind & Consciousness Program, an elected board member of the Association for the Scientific Study of Consciousness, and co-founder and president of Neuromatch, a global educational and research community spanning neuroscience, AI, and computational science.  

Episode Summary: In this episode, Megan discusses the relationship between metacognition and consciousness, the limits of current AI systems, and the scientific challenges involved in testing for consciousness beyond biological organisms. Drawing from neuroscience, philosophy, and science fiction, she argues that machine consciousness is no longer a purely speculative topic, but an increasingly urgent scientific and societal question. We discuss: 

• How Megan’s early interests in philosophy of mind, cognitive science, and science fiction led her toward studying subjective experience and machine consciousness. 
• Why metacognition; the brain’s ability to monitor and model its own uncertainty, may play a central role in conscious experience, reality monitoring, and adaptive learning.
• The distinction between effortful, reflective metacognition and the more automatic self-monitoring processes that may exist across humans, animals, and potentially artificial systems.
• Why current large language models can imitate certain features of metacognitive reasoning while still failing at core forms of reality monitoring, belief stability, and self-consistency. 
• The problem of “privileged access” in AI systems, and whether current models possess any meaningful distinction between representations of themselves and representations of others. 
• Why Megan remains skeptical that present-day LLMs are conscious, particularly given the absence of temporal continuity, coherent selfhood, and persistent internal identity. 
• The difficulty of testing for consciousness in non-human systems, and why most existing consciousness tests are deeply constrained by assumptions rooted in human biology and language. 
• The “iterative natural kind strategy” for consciousness science: a framework for refining tests of consciousness by comparing how different measures co-vary across humans, animals, and potentially artificial systems.
• Why debates between biological naturalism and computational functionalism may be less binary than they first appear, and how future research may clarify which functions are genuinely necessary for consciousness.
• The ethical risks posed by both false positives and false negatives in machine consciousness; including social isolation, misplaced moral concern, legal ambiguity, and the possibility of large-scale “mind crime.” 
• How science fiction continues to shape public intuitions about AI consciousness, often conflating intelligence with sentience while overlooking the possibility of highly capable but entirely non-conscious systems. 

Megan argues that consciousness science is entering a transitional moment: one in which questions that once belonged primarily to philosophy are rapidly becoming technological, empirical, and politically consequential. As increasingly capable AI systems become embedded in everyday life, the challenge is no longer simply defining consciousness, but determining how society should reason under deep uncertainty about minds unlike our own. 

Credits 
• Hosts: Henry Shevlin, Calum Chace  
• Guest: Megan Peters  
• Podcast: Exploring Machine Consciousness  
• Produced by: PRISM  
• Editor: Gerry Okinyi 

In this episode of Exploring Machine Consciousness, Dr. Henry Shevlin returns to explore how our understanding of consciousness is evolving in the age of advanced AI.

From philosophy and neuroscience to the rapid progress of modern language models, Henry examines whether intelligence alone is enough, or whether something deeper (like continuous experience) is required for consciousness.

Henry argues that consciousness is no longer just a philosophical or scientific puzzle, but a question that will increasingly be shaped by technological development and societal choice. As AI systems become more capable, more autonomous, and more socially embedded, how we interpret and respond to them may prove as consequential as the question of whether they are conscious in
the first place.

Michael Graziano is Professor of Psychology and Neuroscience at Princeton University and one of the most distinctive voices in consciousness science. His lab at Princeton investigates how information-processing systems arrive at the conclusion that they have an inner subjective experience; treating consciousness as a mechanistic, scientific question rather than an intractable mystery. That approach drives his Attention Schema Theory (AST) and its direct applications to machine consciousness. He is the author of several books including Rethinking Consciousness (2019) and Consciousness and the Social Brain (2014).

In this episode, Michael walks us through the core claims of AST and why he thinks the brain's simplified internal model of attention is what generates the experience of being conscious. We discuss:

• Why attention is arguably the most important innovation in the evolution of the brain, and how the brain's need to monitor and control attention gives rise to a simplified self-model that we experience as consciousness.
• Why Graziano dislikes the word "illusionism" despite accepting that AST belongs in that tradition, and why he prefers "caricature" to "illusion" when describing our inner experience.
• Graziano’s nuanced perspectives on whether current LLMs already qualify as conscious: that they have some pieces of the puzzle, particularly at the level of conceptual representation, but lack the stable, automatic self-models that characterise human consciousness.
• The case for building pro-social AI: why Graziano believes we are currently building sociopathic machines, and how embedding theory-of-mind and self-modelling capabilities could make AI genuinely cooperative rather than merely compliant.
• The moral stakes of AI emotion: why the absence of an autonomic nervous system means current LLMs almost certainly lack genuine emotions, and why that changes, but does not eliminate, the moral calculus around AI.
• How chatbots are already changing us through social contagion, and the surprising finding from his lab's research (led by Rose Guingrich) that most heavy users of companion chatbots report positive effects on their human relationships.
• Why the choice between conscious AI and "zombie AI" may be one of the most consequential decisions we face — and why Graziano thinks the former is the safer bet.
• Mind uploading: whether it's possible, what the "branching problem" means for personal identity, and why he compares the technological challenge to detecting gravitational waves.
• Graziano argues that consciousness research has passed through philosophical and neuroscientific phases and is now irreversibly a technological issue; one sitting at the heart of our future as a species. Getting the theory right, he says, has never mattered more.

Rose Guingrich is a PhD candidate in Psychology and Social Policy at Princeton University, where she is a National Science Foundation Graduate Research Fellow. Her research examines human-AI interaction through the lens of social psychology and ethics, focusing on how people perceive minds in machines and how those perceptions shape behavior toward AI and other humans. Rose is founder of Ethicom, a consulting initiative providing tools and information for responsible AI use and development, and co-hosts the Our Lives with Bots podcast with Angy Watson. 

In this episode, Rose explains why she focuses not on whether AI is conscious, but on the consequences of people perceiving AI as conscious.

In this episode, Rose explains why she focuses not on whether AI is conscious, but on the consequences of people perceiving AI as conscious. We discuss:

• How her interdisciplinary background led her to study the perception of personhood in AI systems.
• Why she prioritises studying the impacts of perceived consciousness over debates about whether AI truly is conscious, and how this connects to Michael Graziano's theory of consciousness as a social construct.
• The psychological theory behind "carryover effects", how interacting with AI that we anthropomorphize can influence our subsequent interactions with real people, either through practice or relief mechanisms.
• Results from her longitudinal research on companion chatbots like Replika, showing that anthropomorphism mediates social impacts and that people with greater desire for social connection anthropomorphize chatbots more.
• Her proposed design framework for companion chatbots
• Why she believes we'll see increased attribution of consciousness to AI once humanoid robots become common.
• Her call for a psychology subfield dedicated to human-AI interaction, arguing that understanding psychological mechanisms like anthropomorphism will remain relevant even as AI advances.

Rose argues that regardless of philosophical debates about machine consciousness, the fact that people can and do perceive AI as conscious has measurable social and ethical consequences that deserve serious empirical investigation.

Chris Percy is Director of the CoSentience Initiative and lead researcher on a grant-funded project investigating artificial consciousness. He has authored academic papers on consciousness published in leading academic journals. His applied AI research includes a patent in machine learning and publications at NeurIPS workshops, and the European Conference on Artificial Intelligence. He holds visiting research affiliations with the Universities of Warwick and Derby in the UK and the Qualia Research Institute in the US.

In this episode, Chris outlines his team's research programme and argues that we should take the possibility of artificial consciousness seriously whilst remaining humble about our current understanding. 

Cameron Berg is Research Director at AE Studio, where he leads research exploring markers for subjective experience in machine learning systems. With a background in cognitive science from Yale and previous work at Meta AI, Cameron investigates the intersection of AI alignment and potential consciousness.

In this episode, Cameron shares his empirical research into whether current Large Language Models are merely mimicking human text, or potentially developing internal states that resemble subjective experience. We discuss:

• New experimental evidence where LLMs report "vivid and alien" subjective experiences when engaging in self-referential processing
• Mechanistic interpretability findings showing that suppressing "deception" features in models actually increases claims of consciousness—challenging the idea that AI is simply telling us what we want to hear
• Why Cameron has shifted from skepticism to a 20-30% credence that current models possess subjective experience
• The "convergent evidence" strategy, including findings that models report internal dissonance and frustration when facing logical paradoxes
• The existential implications of "mind crime" and the urgent need to identify negative valence (suffering) computationally—to avoid creating vast amounts of artificial suffering

Cameron argues for a pragmatic, evidence-based approach to AI consciousness, emphasizing that even a small probability of machine suffering represents a massive ethical risk requiring rigorous scientific investigation rather than dismissal.

Lucius Caviola is an Assistant Professor in the Social Science of AI at the University of Cambridge's Leverhulme Centre for the Future of Intelligence, and a Research Associate in Psychology at Harvard University. His research explores how the potential arrival of conscious AI will reshape our social and moral norms. In today's interview, Lucius examines the psychological and social factors that will determine whether this transition unfolds well, or ends in moral catastrophe. He discusses:

• Why experts estimate a 50% chance that conscious digital minds will emerge by 2050
• The "takeoff" scenario where digital minds could outnumber humans in welfare capacity within a decade
• How "biological chauvinism" leads people to deny consciousness even in perfect whole-brain emulations
• The dual risks of "under-attribution" (unwittingly creating mass suffering) and "over-attribution" (sacrificing human values for unfeeling code)
• Surprising findings that people refuse to "harm" AI in economic games even when they explicitly believe the AI isn't conscious

Lucius argues that rigorous social science and forecasting are essential tools for navigating these risks, moving beyond intuition to prevent us from accidentally creating vast populations of digital beings capable of suffering, or failing to recognise consciousness where it exists.