The Deepfake Dialogues

In May 2025, a man killed in a road rage shooting in Arizona appeared to address the court at the sentencing of the man who shot him. The video was AI-generated, created by his sister, Stacey Wales, to give him a voice in a process that had reduced him to evidence.

Stacey joins The Deepfake Dialogues to share who her brother Christopher Pelkey was, how her family built the avatar, and why she believes it was worth the risk.

In this episode:

* Who Chris was, and how the courtroom left him faceless and voiceless

* Why Stacey wrote an impact statement in Chris’s voice rather than her own

* How the avatar was built from one photo, a single recorded interview and her script

* The ethical choices behind it, including disclosing it as AI and stopping short of having Chris forgive the shooter

* How the judge and defence responded in court

* The public backlash, and whether anyone can speak for the dead

Generative AI is making scams easier to create, harder to detect, and possible to operate at unprecedented scale. Voice clones can impersonate family members in distress, AI-generated personas can sustain romance scams, and deepfake videos can make trusted public figures appear to promote fraudulent investments.

Ben Winters, Director of AI and Privacy at the Consumer Federation of America, joins The Deepfake Dialogues to discuss how AI is changing the scam landscape—and why treating fraud as a problem of individual gullibility misses the companies and technologies that make these scams possible.

We discuss:

* How voice cloning is transforming “grandparent” and impersonation scams

* How generative AI is making romance scams more convincing and scalable

* Deepfake investment scams and fake celebrity endorsements

* Why older adults are targeted—and why vulnerability cannot be reduced to a lack of tech literacy

* Ben’s concept of the “scam stack,” from data brokers and AI companies to social media platforms and financial services

* Why Meta and other platforms should face greater scrutiny for the scams they help distribute

* The Consumer Federation of America’s lawsuit against Meta over scam advertisements

* What stronger safeguards around voice cloning, payments, data brokers, and scam reporting could look like

* Why scam prevention education matters, but cannot substitute for systemic accountability

A voice is more than a means of communication. It carries our age, identity, personality, and history. For people who cannot speak or are losing their speech, synthetic voice technology can preserve or create something deeply personal: a voice that sounds like their own.

Professor Rupal Patel, a speech scientist at Northeastern University and founder of VocaliD, joins me to trace the evolution of synthetic speech from generic computer voices to highly realistic voice clones. We explore how VocaliD combined the identity carried in a recipient’s vocalisations with the clarity of a donated voice, and how the same technology built to restore voices revealed new risks around impersonation, fraud, and consent.

We discuss:

* Why children and adults who use communication devices were historically given the same generic voices

* How VocaliD created personalised voices for people who had never been able to speak clearly

* The Human Voicebank Initiative and the people who donated their voices

* What it means for someone with ALS or throat cancer to hear a voice that sounds like their own

* How synthetic voices were able to defeat bank voice-authentication systems

* The challenge of safeguarding a technology that can be used for both accessibility and impersonation

* Whether someone’s voice should be recreated after their death

* How changes in the voice could serve as early indicators of Parkinson’s, ALS, and other health conditions

A politician says something inconvenient on live television, in front of a crowd, with a sign-language interpreter beside him. He later claims the footage was made with AI. Nothing was fabricated. The denial alone was enough to muddy the story.

Sam Gregory is Executive Director of WITNESS, the human rights organisation founded in 1992 to help people document what is happening to them and ensure that what they capture is believed. That second task has become increasingly difficult.

Sam has worked on synthetic media since 2018, when WITNESS held some of the first global consultations on deepfakes with journalists and frontline human rights defenders under the banner “prepare, don’t panic.” Those communities warned that deepfakes would not only enable convincing fabrications. They would also give powerful people a new excuse to dismiss real evidence, intensifying a problem that human rights defenders had faced for decades. This conversation explores how we can continue to establish the truth when authentic footage can so easily be denied.

We discuss:

* Why fabrication and denial can now occur around the same event, and sometimes the same image

* Detection equity: why accent, skin tone and camera quality can affect how reliably a deepfake is identified

* What the Deepfakes Rapid Response Force does when contested media surfaces during a crisis

* Why courts are approaching a reckoning with “prove it was me” defences

* Where provenance standards such as C2PA genuinely help, and how they could be weaponised

* Why “spot the six fingers” was always the wrong lesson to teach

Anyone’s face, voice, or name can now be synthesised in seconds. So how do you protect a person’s likeness when the unauthorised version can be created from scratch, no manipulation required?

Dan Neely, co-founder and CEO of Vermillio and one of TIME’s 100 Most Influential People in AI, is building the guardrails for the generative internet. Vermillio’s TraceID technology creates a neural fingerprint of a person’s likeness and scours the web to detect when their face, voice, or name has been used to generate AI content, whether that’s a deepfake designed to deceive or an unauthorised AI track. Beyond detection, the company is building the infrastructure for authorised AI licensing, so creators can control and even monetise how their likeness is used.

We discuss:

* How TraceID fingerprints a likeness and matches it against a trillion pieces of AI content a month

* Why attribution goes further than simply asking “is this AI or not?”

* Why likeness data is “strategic gold” creators shouldn’t hand over to big tech

* Where the line sits between an unauthorised deepfake and legitimate parody or fan content

* What happens when a model trained in a country with no IP laws uses your face

While Congress stalls, state legislatures have quietly become the front line in the fight against deepfakes. This episode features Ilana Beller, who leads Public Citizen’s state legislative work on AI and has helped move model legislation through more than 40 states, focused on the two areas moving fastest: non-consensual intimate deepfakes and election deepfakes.

We discuss:

* What the Take It Down Act accomplished and left on the table, including the missing private right of action

* How intimate deepfake laws work, and why circulation is easier to prosecute than creation

* What separates a strong law from a weak one: enforcement and a high bar for consent

* App-level bans on nudification tools and barring platforms from advertising them

* How election deepfake laws use labeling and pre-election windows while carving out satire and parody

* Why these bills draw rare bipartisan support, and the federal moratorium threatening to preempt them

What happens when a local councillor becomes the target of a deepfake smear campaign?

In March 2026, a deepfake video of Cllr Pete Kilbane, Labour Deputy Leader of City of York Council, began circulating online. In the clip, his digitally fabricated likeness appeared to hand wads of cash to balaclava-clad men, as if paying them to remove flags. The video was fake. The campaign of intimidation surrounding it was not.

In this episode, Pete walks through what happened in York after St George’s Cross and Union flags began appearing on lampposts across the city as part of Operation Raise the Colours. Hope Not Hate and other public reporting have linked the movement to figures associated with Tommy Robinson and the far-right party Britain First.

We discuss:

* How the flag campaign played out in York and why the politics behind it matter

* The deepfake video of Pete, how his colleagues first found out, and why “it’s just satire” does not capture the harm

* The threats, doxxing, and intimidation faced by council staff and contractors

* What the Online Safety Act can and cannot do when fake content spreads through closed groups

* Pete’s concerns about the movement’s links to MAGA and the American far right

* How synthetic media is being weaponised to erode trust in local democracy

* How the council responded to this incident

Content note: This episode discusses AI-generated child sexual abuse material, non-consensual intimate imagery, sextortion, and sexual abuse.

AI-powered “nudify” apps are increasingly being used by students to create deepfake sexual images of classmates, but most schools still do not have clear policies for prevention, reporting, discipline, or victim support.

Riana Pfefferkorn, a policy fellow at Stanford HAI, joins DeepFake Dialogues to discuss her policy brief, Addressing AI-Generated Child Sexual Abuse Material: Opportunities for Educational Policy, and what schools, lawmakers, platforms, and communities need to understand about how this is actually playing out.

Read the policy brief here: https://hai.stanford.edu/policy/addressing-ai-generated-child-sexual-abuse-material-opportunities-for-educational-policy

We discuss:

* The difference between fully synthetic AI-CSAM and images depicting real, identifiable children

* The lasting harm to victims, and why it often doesn’t end at graduation

* How unprepared school responses can make that harm worse

* What schools should have in place before an incident happens

* Whether student offenders should face criminal punishment or behavioural intervention

* The role of “nudify” app bans, platform liability, and payment processors

* Grok, xAI, and what happens when mainstream AI tools enable sexualised image abuse

* Why prevention has to include consent, dignity, privacy, and digital citizenship

AI-generated wildlife videos are flooding social media, and the consequences go far beyond misleading content.

Dana Wilson of Wildlife SOS, an Indian nonprofit running 13 rescue centers and emergency hotlines across the country, explains how synthetic animal videos are distorting public understanding of wildlife behavior, putting both people and animals at risk.

We discuss:

* How Wildlife SOS’s real rescue footage has been stolen and spliced with AI-generated endings

* The public safety threat of videos that make dangerous predators look friendly and approachable

* AI-generated attack videos stoking fear and calls for culling in communities that coexist with wildlife

* Why even “harmless” AI-generated animal content erodes trust in genuine conservation storytelling

* How deepfake technology could make these problems far worse as synthetic video becomes harder to detect

* How Wildlife SOS is setting internal AI policies to protect the integrity of their work

* What creators and platforms should consider before posting AI-generated wildlife content

Learn more about Wildlife SOS at wildlifesos.org and watch real rescue videos on their YouTube channel.

In this episode, I’m joined by Erin D. Dumbacher, Stanton Nuclear Security Senior Fellow at the Council on Foreign Relations, to discuss her recent Foreign Affairs article “How Deepfakes Could Lead to Doomsday.” We explore how deepfakes and AI hallucinations could infiltrate the systems designed to prevent nuclear catastrophe — and why the story of Stanislav Petrov, a Soviet officer who in 1983 chose not to trust his screen when it told him American nukes were incoming, matters more now than ever.

We also dig into the Pentagon’s rush to integrate AI across the Department of Defense, why AI trained on nuclear threats has almost no real data to learn from, and why the most dangerous scenario isn’t a machine launching a weapon — it’s bad information cascading through a decision-making chain where the president has minutes, not hours, to act.

We discuss:

* Deepfakes of Zelensky and Putin during the Ukraine war — and what could have gone differently

* The Pentagon’s GenAI.mil rollout and the risks of moving too fast

* AI hallucinations vs. social media deepfakes — two distinct threats to nuclear stability

* The data problem: training AI when we’ve only had two nuclear attacks in history

* How misinformation could trigger cascading crises under compressed nuclear timelines

* Reforms to U.S. nuclear launch authority in an era where information can’t be taken at face value