In this episode of Conversations with Zena, My AI Colleague, host David Espindola is joined by Dr. Craig Kaplan, a pioneer in artificial intelligence whose career spans more than four decades. From his early work at Carnegie Mellon alongside Nobel laureate Herbert Simon to his current focus on AI safety and ethics, Craig has spent his career thinking deeply about how intelligent systems evolve—and how they can be designed to remain aligned with human values. He is also the founder of Superintelligence.com, a domain he secured nearly two decades ago, long before the topic entered mainstream discourse.

The conversation begins by grounding listeners in the evolution of AI—from narrow, task-specific systems to artificial general intelligence (AGI), and ultimately to superintelligence: systems that could outperform even the most capable humans across nearly all cognitive domains. Craig explains why timelines once measured in decades are now being compressed into years, and why this acceleration has surprised even seasoned AI researchers.

From there, the discussion turns to one of the most consequential questions of our time: Can superintelligent systems be aligned with human values? David references the concerns raised by thinkers like Nick Bostrom, who argue that controlling a vastly superior intelligence may be impossible. Craig agrees that traditional notions of “control” are insufficient—but reframes the challenge as a design problem rather than a control problem.

Instead of ever-larger, opaque “black box” models, Craig advocates for a collective or democratic intelligence approach: systems composed of many interacting AI agents operating with transparency, checks and balances, and shared rules. Drawing on real-world experience building collective intelligence systems that outperformed elite Wall Street investors, he argues that intelligence can emerge from coordination—not centralization—and that such systems can be both more capable and safer.

The conversation deepens when Zena, David’s AI co-host trained on his work and values, joins in. Together they explore how personalized AI agents—each reflecting the values of their human counterpart—could create a more representative, accountable, and ethically grounded AI ecosystem, rather than concentrating moral authority in the hands of a few organizations.

A pivotal moment comes with Craig’s explanation of “P(doom)”—the estimated probability that advanced AI could pose an existential risk to humanity. Citing discussions with leading AI researchers, Craig notes that many place this risk between 10% and 20%, a number far too high to ignore. Rather than calling for pauses or relying solely on regulation, he argues for architectures that are safer by design—and that can still move fast and create economic value.

The episode closes with a powerful reminder: we are all already shaping the future of AI. Every online action contributes to the data that trains these systems. In that sense, humanity is already teaching AI how to behave—often without realizing it.

This is a thoughtful, balanced exploration of superintelligence that takes risk seriously without surrendering to fear, and offers a hopeful vision for how humans and AI might evolve together through transparency, accountability, and shared values.

Opening to Conversations with Zena.

Music at the the end of each episode

Support the show