Katja Grace joins the podcast to discuss the largest survey of AI researchers conducted to date, AI researchers' beliefs about different AI risks, capabilities required for continued AI-related transformation, the idea of discontinuous progress, the impacts of AI from either side of the human-level intelligence threshold, intelligence and power, and her thoughts on how we can mitigate AI risk. Find more on Katja's work at https://aiimpacts.org/.

Timestamps:

0:20 AI Impacts surveys

18:11 What AI will look like in 20 years

22:43 Experts’ extinction risk predictions

29:35 Opinions on slowing down AI development

31:25 AI “arms races”

34:00 AI risk areas with the most agreement

40:41 Do “high hopes and dire concerns” go hand-in-hand?

42:00 Intelligence explosions

45:37 Discontinuous progress

49:43 Impacts of AI crossing the human-level intelligence threshold

59:39 What does AI learn from human culture?

1:02:59 AI scaling

1:05:04 What should we do?

Holly Elmore joins the podcast to discuss pausing frontier AI, hardware overhang, safety research during a pause, the social dynamics of AI risk, and what prevents AGI corporations from collaborating. You can read more about Holly's work at https://pauseai.info

Timestamps:

00:00 Pausing AI

10:23 Risks during an AI pause

19:41 Hardware overhang

29:04 Technological progress

37:00 Safety research during a pause

54:42 Social dynamics of AI risk

1:10:00 What prevents cooperation?

1:18:21 What about China?

1:28:24 Protesting AGI corporations

Sneha Revanur joins the podcast to discuss the social effects of AI, the illusory divide between AI ethics and AI safety, the importance of humans in the loop, the different effects of AI on younger and older people, and the importance of AIs identifying as AIs. You can read more about Sneha's work at https://encodejustice.org

Timestamps:

00:00 Encode Justice

06:11 AI ethics and AI safety

15:49 Humans in the loop

23:59 AI in social media

30:42 Deteriorating social skills?

36:00 AIs identifying as AIs

43:36 AI influence in elections

50:32 AIs interacting with human systems

Roman Yampolskiy joins the podcast again to discuss whether AI is like a Shoggoth, whether scaling laws will hold for more agent-like AIs, evidence that AI is uncontrollable, and whether designing human-like AI would be safer than the current development path. You can read more about Roman's work at http://cecs.louisville.edu/ry/

Timestamps:

00:00 Is AI like a Shoggoth?

09:50 Scaling laws

16:41 Are humans more general than AIs?

21:54 Are AI models explainable?

27:49 Using AI to explain AI

32:36 Evidence for AI being uncontrollable

40:29 AI verifiability

46:08 Will AI be aligned by default?

54:29 Creating human-like AI

1:03:41 Robotics and safety

1:09:01 Obstacles to AI in the economy

1:18:00 AI innovation with current models

1:23:55 AI accidents in the past and future

On this special episode of the podcast, Flo Crivello talks with Nathan Labenz about AI as a new form of life, whether attempts to regulate AI risks regulatory capture, how a GPU kill switch could work, and why Flo expects AGI in 2-8 years.

Timestamps:

00:00 Technological progress

07:59 Regulatory capture and AI

11:53 AI as a new form of life

15:44 Can AI development be paused?

20:12 Biden's executive order on AI

22:54 How would a GPU kill switch work?

27:00 Regulating models or applications?

32:13 AGI in 2-8 years

42:00 China and US collaboration on AI

Carl Robichaud joins the podcast to discuss the new nuclear arms race, how much world leaders and ideologies matter for nuclear risk, and how to reach a stable, low-risk era. You can learn more about Carl's work here: https://www.longview.org/about/carl-robichaud/

Timestamps:

00:00 A new nuclear arms race

08:07 How much do world leaders matter?

18:04 How much does ideology matter?

22:14 Do nuclear weapons cause stable peace?

31:29 North Korea

34:01 Have we overestimated nuclear risk?

43:24 Time pressure in nuclear decisions

52:00 Why so many nuclear warheads?

1:02:17 Has containment been successful?

1:11:34 Coordination mechanisms

1:16:31 Technological innovations

1:25:57 Public perception of nuclear risk

1:29:52 Easier access to nuclear weapons

1:33:31 Reaching a stable, low-risk era

Frank Sauer joins the podcast to discuss autonomy in weapon systems, killer drones, low-tech defenses against drones, the flaws and unpredictability of autonomous weapon systems, and the political possibilities of regulating such systems. You can learn more about Frank's work here: https://metis.unibw.de/en/

Timestamps:

00:00 Autonomy in weapon systems

12:19 Balance of offense and defense

20:05 Killer drone systems

28:53 Is autonomy like nuclear weapons?

37:20 Low-tech defenses against drones

48:29 Autonomy and power balance

1:00:24 Tricking autonomous systems

1:07:53 Unpredictability of autonomous systems

1:13:16 Will we trust autonomous systems too much?

1:27:28 Legal terminology

1:32:12 Political possibilities

Darren McKee joins the podcast to discuss how AI might be difficult to control, which goals and traits AI systems will develop, and whether there's a unified solution to AI alignment.

Timestamps:

00:00 Uncontrollable superintelligence

16:41 AI goals and the "virus analogy"

28:36 Speed of AI cognition

39:25 Narrow AI and autonomy

52:23 Reliability of current and future AI

1:02:33 Planning for multiple AI scenarios

1:18:57 Will AIs seek self-preservation?

1:27:57 Is there a unified solution to AI alignment?

1:30:26 Concrete AI safety proposals

Mark Brakel (Director of Policy at the Future of Life Institute) joins the podcast to discuss the AI Safety Summit in Bletchley Park, objections to AI policy, AI regulation in the EU and US, global institutions for safe AI, and autonomy in weapon systems.

Timestamps:

00:00 AI Safety Summit in the UK

12:18 Are officials up to date on AI?

23:22 Objections to AI policy

31:27 The EU AI Act

43:37 The right level of regulation

57:11 Risks and regulatory tools

1:04:44 Open-source AI

1:14:56 Subsidising AI safety research

1:26:29 Global institutions for safe AI

1:34:34 Autonomy in weapon systems

Dan Hendrycks joins the podcast again to discuss X.ai, how AI risk thinking has evolved, malicious use of AI, AI race dynamics between companies and between militaries, making AI organizations safer, and how representation engineering could help us understand AI traits like deception. You can learn more about Dan's work at https://www.safe.ai

Timestamps:

00:00 X.ai - Elon Musk's new AI venture

02:41 How AI risk thinking has evolved

12:58 AI bioengeneering

19:16 AI agents

24:55 Preventing autocracy

34:11 AI race - corporations and militaries

48:04 Bulletproofing AI organizations

1:07:51 Open-source models

1:15:35 Dan's textbook on AI safety

1:22:58 Rogue AI

1:28:09 LLMs and value specification

1:33:14 AI goal drift

1:41:10 Power-seeking AI

1:52:07 AI deception

1:57:53 Representation engineering