Send us a text

What happens when we outsource our thinking to intelligent machines? Kyle Killian, an expert in AI security, joins the conversation to unpack the critical distinctions between AI alignment, security, and safety—and why these differences matter for our collective future.

The dialogue begins by clarifying these often-confused terms: alignment aims to make AI systems follow human intentions, while security focuses on protecting AI systems from threats and preventing misuse. As Kyle explains, "AI security is more focused on learning how to actually secure systems from competitors and adversaries."

We explore the concerning phenomenon of "human enfeeblement"—our growing dependency on algorithmic systems that gradually erodes our cognitive capabilities. From navigation apps to complex decision-making, we're increasingly outsourcing mental tasks to machines. "Before you know it, it's running larger and larger portions of your life," Kyle warns, describing how we become "cyborgs with our cell phones" in this slow boiling frog scenario.

Particularly alarming is the re-emergence of reinforcement learning in advanced AI systems, bringing back concerns about power-seeking behaviors and specification gaming that earlier safety researchers identified. This technical shift has profound implications for safety, as these models can develop unexpected strategies to achieve their programmed goals.

The conversation takes a sobering turn when examining AI's potential role in military applications. Research shows AI systems tend toward escalation in conflict simulations, lacking human restraint when faced with high-stakes decisions. "They didn't have the same reflection on these catastrophic consequences," Kyle notes, highlighting the dangers of incorporating AI into defense systems, particularly nuclear command and control.

Despite these serious concerns, we conclude by searching for positive paths forward, emphasizing the need for global cooperation on AI governance and the importance of clear communication about risks. As Kyle suggests, meaningful progress requires "understanding what we're talking about so we can get rid of any bias or miscommunications" that currently hamper safety efforts.

How will you navigate this rapidly evolving landscape where machines increasingly think alongside—and potentially for—us?

Support the show