Humans On The Loop

Design for Provably Safe AI with Evan Miyazono


Listen Later

Membership | Donations | Spotify | YouTube | Apple Podcasts

This week’s guest is my friend Evan Miyazono, CEO and Director of Atlas Computing — a tech non-profit committed not to the false god of perfect alignment but to plausible strategy of provable safety. Focusing on community building, cybersecurity, and biosecurity, Evan and his colleagues are working to advance a new AI architecture that constrains and formally specifies AI outputs, with reviewable intermediary results, collaborating across sectors to promote this radically different and more empirical approach to applied machine intelligence.

After completing his PhD in Applied Physics at Caltech, Evan led research at Protocol Labs, creating their research grants program, and led the special projects team that created Hypercerts, Funding the Commons, gov4git, and key parts of Discourse Graphs and the initial Open Agency Architecture proposal.

In our conversation we talk about a wide swath of topics including regulatory scaling problems, specifying formal organizational charters, the spectre of opacity, and the quantification of trust — all, in some sense, interdisciplinary matters of “game design” in our entanglement with magical technologies and fundamental uncertainty.

If you enjoy this conversation, join the Wisdom x Technology Discord server and consider becoming a member for access to our study groups, community calls, and complete archives.

Founding members also get access to the entire twenty hours of lecture and discussion from my recent course, How to Live in the Future.

Links

Hire me for speaking or consulting• Explore the Humans On The Loop archives• Dig into nine years of mind-expanding podcasts• Browse the books we discuss on the show at Bookshop.org• Explore the interactive knowledge garden grown from over 250 episodes

Discussed

• Atlas Computing Summary Slides• Atlas Computing Institute Talks (YouTube Playlist)• A Toolchain for AI-Assisted Code Specification, Synthesis and Verification• Also, a relevant paper from Max Tegmark:Provably safe systems: the only path to controllable AGI

Mentioned

Gregory BatesonDavid DalrympleK. Allado-McDowellTerence McKennaYuval Noah HarariCosma ShaliziHenry FarrellHakim BeyNatalie DeprazFrancisco VarelaPierre VermerschPlurality InstitutePuja OhlhaverSean Esbjörn-HargensAlfred North WhiteheadDe Kai

Primer Riff

Are we doing AI alignment wrong? Game designers Forrest Imel and Gavin Valentine define games as having meaningful decisions, uncertain outcomes, and measurable feedback. If any one of these breaks, the game breaks. And we can think about tech ethics through this lens as well. Much of tech discourse is about how one or more of these dimensions has broken the “game” of life on Earth — the removal of meaningful decisions, the mathematical guarantee of self-termination through unsustainable practices, and/or the decoupling of feedback loops.

AI alignment approaches tend to converge on restoring meaningful decisions by getting rid of uncertainty, but it’s a lost cause. It’s futile to encode our values into systems we can’t understand. To the extent that machines think, they think very differently than we do, and characteristically “interpret” our requests in ways that reveal the assumptions we are used to making based on shared context and understanding with other people.

We may not know how a black box AI model arrives at its outputs, but we can evaluate those outputs…and we can segment processes like this so that there are more points at which to review them. One of this show’s major premises is that the design and use of AI systems is something like spellcraft — a domain where precision matters because the smallest deviation from a precise encoding of intent can backfire.

Magic isn’t science in as much as we can say that for spellcraft, mechanistic understanding is, frankly, beside the point. Whatever you may think of it, spellcraft evolved as a practical approach for operating in a mysterious cosmos. Westernized Modernity dismisses magic because Enlightenment era thinking is predicated on the knowability of nature and the conceit that everything can and will eventually bend to principled, rigorous investigation. But this confused accounting just reshuffled its uneradicable remainder of fundamental uncertainty back into a stubbornly persistent Real that continues to exist in excess of language, mathematics, and mechanistic frameworks. Economies, AI, and living systems guarantee uncertain outcomes — and in accepting this, we have to re-engage with magic in the form of our machines. The more alike they become, the more our mystery and open-ended co-improvisation loom back over any goals of final knowledge and control.

In a 2016 essay, Danny Hillis called this The Age of Entanglement. It is a time that calls for an evolutionary approach to technology. Tinkering and re-evaluating, we find ourselves one turn up the helix in which quantitative precision helps us reckon with the new built wilderness of technology. When we cannot fully explain the inner workings of large language models, we have to step back and ask:

What are our values, and how do we translate them into measurable outputs?

How can we break down the wicked problem of AI controllability into chunks on which it’s possible to operate?

How can adaptive oversight and steering fit with existing governance processes?

In other words, how can we properly task the humanities with helping us identify “meaningful decisions” and the sciences with providing “measurable feedback.” Giving science the job of solving uncertainty or defining our values ensures we’ll get as close as we can to certitude about outcomes we definitely don’t want. But if we think like game designers, then interdisciplinary collaboration can help us safely handle the immense power we’ve created and keep the game going.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit michaelgarfield.substack.com/subscribe
...more
View all episodesView all episodes
Download on the App Store

Humans On The LoopBy Michael Garfield

  • 4.9
  • 4.9
  • 4.9
  • 4.9
  • 4.9

4.9

242 ratings


More shows like Humans On The Loop

View all
Tangentially Speaking with Christopher Ryan by Chris Ryan

Tangentially Speaking with Christopher Ryan

2,063 Listeners

Rune Soup by Gordon

Rune Soup

759 Listeners

THIRD EYE DROPS by Michael Phillip

THIRD EYE DROPS

1,242 Listeners

Team Human by Douglas Rushkoff

Team Human

359 Listeners

Living Myth by Michael Meade

Living Myth

967 Listeners

Weird Studies by SpectreVision Radio

Weird Studies

588 Listeners

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas by Sean Carroll | Wondery

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

4,127 Listeners

New Thinking Allowed Audio Podcast by New Thinking Allowed Audio Podcast

New Thinking Allowed Audio Podcast

255 Listeners

Hermitix by Hermitix

Hermitix

342 Listeners

The Emerald by Joshua Schrei

The Emerald

956 Listeners

The Jim Rutt Show by The Jim Rutt Show

The Jim Rutt Show

256 Listeners

Acid Horizon by Acid Horizon

Acid Horizon

177 Listeners

Theories of Everything with Curt Jaimungal by Theories of Everything

Theories of Everything with Curt Jaimungal

469 Listeners

The Great Simplification with Nate Hagens by Nate Hagens

The Great Simplification with Nate Hagens

396 Listeners

Dr. John Vervaeke by Upfire Digital LLC

Dr. John Vervaeke

62 Listeners