Concerning AI | Existential Risk From Artificial Intelligence

We haven’t found strong arguments on the “Don’t be worried. Here’s why …” side of things. We know the arguments must exist, but we can’t find them (send them to us!). So, what to do? Make some arguments up, that’s what!

Random notes we used to organize our thoughts, not necessarily in the episode:

Jerry Kaplan … we are framing AI wrong.  The main reason people build AI software is to automate tasks, not to create general intelligence.  Therefore we are going to have some economic disruptions (Humans Need Not Apply).  Asserts that worrying about the robot apocalypse is missing the point of what AI is about. “A reminder that AI systems don’t need red laser eyes to be dangerous.”

Moore’s law is over.  All exponentials die.  We’ve never yet seen one that doesn’t come to an end.

All progress comes from the human population explosion and that is over, so the rate of progress is actually slowing down.

There will be many and they will keep each other in check.  This one only helps us if we assume there are other intelligences in the universe and that the AI from here won’t be able to take over everybody.

They could but they won’t want to:

They fragment when they get too big, so they have less incentive to take over everything.

They could use all of our matter without disorganizing us.

Ted: It will be us.

Brandon: They fragment when they get too big.

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

No real notes this week as we have very little idea what we’re talking about.

http://www.kurzweilai.net/why-evolution-may-be-intelligent-based-on-deep-learning

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

As usual, here are the notes we used to create the show. We talked about some of this during this episode:

* OpenAI … wow!

* Exponentials … we don’t think in them

* Moore’s law

Punchline … Bacteria in the bottle.

This is really good! This Week in Startups – Audio: E598: Babak Hodjat, Sentient.ai Co-Founder & Chief Scientist (& main inventor of tech behind Siri), is creating artificial intelligence that learns and adapts with our every decision

Let’s just put our heads in the sand!

Exponentials

** 15 years to sequence HIV, 31days to sequence sars “People declared the Human Genome Project a failure halfway through, when one percent had been decoded — they said it’d take 700 years. My reaction was, no, we’re almost done. It was finished seven years later.”

* also animated gif showing how long it appears nothing happens and then the lake fills up all at once

* Chess board, if $1 on first square, $1 on 2nd, we fill the board with $64. Change the growth rate from 0 to 5% and the board now holds $434. At 20%: $584,000. At 50%: $372 billion. With rice instead of money, doubling every square: 18 quintillion, 446 quadrillion, 744 trillion, 073 billion, 709 million, 551 thousand, 615 – enough rice to pile up to the size of Mt. Everest.

=== Emperor rice on chessboard (wheat on chessboard) ===

* 18 * 1 million * 1 million * 1 million … 18,446,744,073,709,551,615

“The second half of the chessboard” is a Kurzweil phrase that references the point where an exponentially growing factor begins to have significant economic impact on an organizations’s overall business strategy (wikipedia).

India’s annual rice output is about 1.2 million times as large as the total for the first half of the chessboard (4,294,967,295 … 2^32 – 1).  Around 1,000 times the global production of rice in 2010.

=== Moore’s Law ===

* Moore’s second law.  The cost of a semiconductor fab also increases exponentially over time.

=== Sigmoid (Logistic) function ===

1/(1 + e^-t)

=== Kurzweil ==

* My view is that the likely outcome is that on the one hand, from the perspective of biological humanity, these superhuman intelligences will appear to be their transcendent servants, satisfying their needs and desires. On the other hand, fulfilling the wishes of a revered biological legacy will occupy only a trivial portion of the intellectual power that the Singularity will bring.

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

We were a little thin on homework this episode. Here were some thoughts we thought we might start from:

Hugo … cosmists, terrans, and cyborgists (transhumanists are one flavor of C or the other)

Eliezer and Nick … have a framework for thinking about AI ethics

Kurzweil … when the machines say that they are people we’ll believe them

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

Eric Samur’s comment on episode 0005:

Fascinating. It occurred to me that we may find computational super intelligence too complicated to be useful but once we create super intelligences they may not find directly programming even greater intelligences that difficult. It may not so much be a matter of which path as what order and how many iterations (segments) of each path lead from us to the ultimate intelligence (whatever that is).

Evan Prodromou’s comment after episode 0004:

So, Brandon CS Sanders and Ted Ernst Sarvata: I’m loving “Concerning AI”. It’s a great show, and you two have great rapport and cover the issue so well.

There’s one thing I find lacking, which is something I have come to expect from both of you: compassion. If we are able to develop artificial intelligences with a level of autonomy and reasoning equivalent to humans, we have a moral imperative to treat them as people. They would be, by definition, people. Their well-being should factor into our planning, the same way we’d treat extraterrestrial non-human intelligences who shared our planet with us.

What kind of future do we have if we intentionally cripple or enslave people because we are afraid of their potential? If that’s the best we can hope for, what does that say about us as a species?

Maybe the most important thing for humans to be doing in these pre-AI years is to develop legal and ethical frameworks for dealing with non-human people. How smart do you have to be to exercise the rights and freedoms that humans have? Are there other thresholds you have to cross? What does it mean for a non-human person to take their place as an equal individual in our society? And how can we humans learn to welcome our fellow intelligences with dignity and generosity?

Anyways, great show, and love to listen.

As usual, our prep notes don’t necessarily match the content of the episode, but here they are:

Compassion from Ignorance

Click to access artificial-intelligence.pdf

* Exotic properties of mind

* John Rawls … Original Position and the Veil of Ignorance

* The role of Empathy in considering the question

Cosmist vs. Terran

Tim Urban’s (Wait But Why) talk THE AI REVOLUTION: THE ROAD TO SUPERINTELLIGENCE

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

– 3 paths … Connectionist (Neural nets), Computationalist (Symbolic), Robotics (embodied) … Uploading … big data … other paths

– IBM paper says: Silver Bullets, Core Values, Emergence, and Emulation

– Wikipedia page for Bostrom’s book Superintelligence

AI

– connectionism and neural nets

Whole brain emulation

Human-machine interfaces

Organizational

Biological (eg embryo selection, gene hacking)

Game AIs, worlds getting more and more rich

next episode: ethics, how will we deal with emerging non-biological persons

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

Our notes compiled in prep for recording this episode. Much of this we didn’t talk about.

Outline

– nanotechnology

– Talk about expected value of future generations

Ep 0004

Cambridge Center for Study of Existential Risk http://cser.org says “greatest threats” to the human species are man-made; they are artificial intelligence, global warming, nuclear war and rogue biotechnology.

Future of humanity institute gave 19% chance of human extinction before 2100

Nick Bostrom paraphrased … how many other people are already studying it?  Because the more people studying it the less difference a few extra minds make studying it.

From 80,000 hours … in brief, assess causes using the following four factors:

Low hanging fruit.

Examples of well studied topics:

http://www.existential-risk.org/faq.html

Bostrom … would you have warning https://www.youtube.com/watch?v=h9LaSfq64E8

Bostrom … survey … median 50% chance at 2040 or 2050 (depending on group) … https://www.youtube.com/watch?v=aheywElK_U8

2013 Martin Rees

2002 Nick Bostrom

Click to access risks.pdf

– scope, intensity, and probability

– nanotechnology

2008 Eliezer Yudkowsky

Click to access AIPosNegFactor.pdf

Hawking’s AMA Answers (wrt AI)

– It’s clearly possible for a something to acquire higher intelligence than its ancestors: we evolved to be smarter than our ape-like ancestors, and Einstein was smarter than his parents. The line you ask about is where an AI becomes better than humans at AI design, so that it can recursively improve itself without human help. If this happens, we may face an intelligence explosion that ultimately results in machines whose intelligence exceeds ours by more than ours exceeds that of snails.

– The latter. There’s no consensus among AI researchers about how long it will take to build human-level AI and beyond, so please don’t trust anyone who claims to know for sure that it will happen in your lifetime or that it won’t happen in your lifetime. When it eventually does occur, it’s likely to be either the best or worst thing ever to happen to humanity, so there’s huge value in getting it right. We should shift the goal of AI from creating pure undirected artificial intelligence to creating beneficial intelligence. It might take decades to figure out how to do this, so let’s start researching this today rather than the night before the first strong AI is switched on.

– An AI that has been designed rather than evolved can in principle have any drives or goals. However, as emphasized by Steve Omohundro, an extremely intelligent future AI will probably develop a drive to survive and acquire more resources as a step toward accomplishing whatever goal it has, because surviving and having more resources will increase its chances of accomplishing that other goal. This can cause problems for humans whose resources get taken away.

– You’re right that we need to avoid the temptation to anthropomorphize and assume that AI’s will have the sort of goals that evolved creatures to. An AI that has been designed rather than evolved can in principle have any drives or goals. However, as emphasized by Steve Omohundro, an extremely intelligent future AI will probably develop a drive to survive and acquire more resources as a step toward accomplishing whatever goal it has, because surviving and having more resources will increase its chances of accomplishing that other goal. This can cause problems for humans whose resources get taken away.

audio at the end about terrifying AI:

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

This is stuff we thought / read about in preparation for the episode. We didn’t end up talking about all of this.

There are two different senses of skeptic

Terms

Readings:

Jaron Lanier … “It’s true, in some far future situation, we’re going to transition. . . . I think it’s very hard to predict a year.” Approximately when? “I think we’re in pretty safe territory if we say it’s within this century.”

Paul Allen … wrote “the singularity isn’t near” article … “By the end of the century,” he wrote, “we will still be wondering if the singularity is near.”

“I think we are going to start to interconnect as a human species in a fashion that is intimate and magical,” Diamandis told me. “What I would imagine in the future is a meta-intelligence where we are all connected by the Internet [and] achieve a new level of sentience. . . . Your readers need to understand: It’s not stoppable. It doesn’t matter what they want. It doesn’t matter how they feel.”

Charlie Stross … This is my take on the singularity: we’re not going to see a hard take-off, or a slow take-off, or any kind of AI-mediated exponential outburst. What we’re going to see is increasingly solicitous machines defining our environment — machines that sense and respond to our needs “intelligently”. But it will be the intelligence of the serving hand rather than the commanding brain, and we’re only at risk of disaster if we harbour self-destructive impulses.

More Charlie Stross … Any way you cut these three ideas, they don’t provide much in the way of referent points for building a good life, especially if they turn out to be untrue or impossible (the null hypothesis). Therefore I conclude that, while not ruling them out, it’s unwise to live on the assumption that they’re coming down the pipeline within my lifetime.

John Horgan … neuroscience appears to be making “antiprogress” — the more information we acquire, the less we seem to know.

Yann LeCun, the head of Facebook’s AI lab, summed it up in a Google+ post back in 2013: “Hype is dangerous to AI. Hype killed AI four times in the last five decades. AI Hype must be stopped.”

trying to balance the need to start some preliminary work on a threat that looms way off in the distance versus the risk of engendering so much hype that it starts a giant backlash.

John McCarthy once told me that when people ask him when human level AI will be achieved he says between five and five hundred years from now.

When I read the articles about skeptics, I see them making two points over and over again. First, we are nowhere near human-level intelligence right now, let alone superintelligence, and there’s no obvious path to get there from here. Second, if you start demanding bans on AI research then you are an idiot.

The difference between skeptics and believers isn’t about when human-level AI will arrive, it’s about when we should start preparing.

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

Following up on last episode’s question, “What is intelligence?” this episode we attempt to unpack “consciousness.”

We started from a dictionary definition: the state of being awake and aware of one’s surroundings

We then explored a lot of ideas without necessarily making much, if any progress.

In conclusion we decided to take Neil deGrasse Tyson’s lead when he wondered if we’re asking the wrong question about consciousness:

In other words, we punted on consciousness for now.

See also: Wait But Why: Putting Time In Perspective

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/

Our goal on this episode is to establish a “good enough” definition of intelligence that we’ll be able to refer back to in future episodes.

references we used:

Concerning AI | Existential Risk From Artificial Intelligence is made with Jetpack Podcast. Full show notes and every episode at https://concerning.ai/