For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.

Re7: Artifactual Goals

On goals producing artifacts-producing-goals, and machines coming alive.

Air date: Monday, 26th Oct. 2020, 5 : 50 PM Pacific/US.

1 Goals produce artifacts

Simon says something like this, and that goals produce more goals.1

Artifacts might absorb goals

It really depends on the physical interpretation of 'goal': does an idea, or intention, leave the mind, or become somehow embodied or instantiated or imitated in, say, an abacus?

Artifacts cause goals

A constructed abacus often causes the goal of displaying that abacus.

Artifacts might absorb their own goals

Any machines that move, that have moving parts, have something like goal absorption and exsorption happening between the parts inside them.

Artifacts might absorb each others' goals

Using two abaci together, might a goal move from one to the other via you?

Machines alive

The animated behavior of machines suggests a question: At what point do 'inanimate' things start having their own goals? Consider AlphaZero, managing Google data center power supplies.2

Consider SCADA systems, or nuclear missile silos, which have so much automation (animation) built into them.

Goals coming out of machines

This is a well-worn topic.3

The goal is the key. If there is something about goals that we don't understand, that fact might be good news or bad news about the future of AI (and humanity).

Artifacts exchanging goals

If goals can move between entities of any sort, they might move between entities of all sorts.

2 What would we do if we knew?

If we knew whether machines can have goals, or share them, or become lifelike, or become dangerous, what action, on our part, would that imply? It's not obvious.

How could the world be reorganized to prevent dangerous super AI?

Is it something we're doing, or something that's happening to us?

Totalitarianism

Totalitarianism is one of the things Nick Bostrom considers.4

War

Hugo de Garis predicts an 'artilect war' between 'cosmists' (people bent on building super-AI, no matter what the consequences) and 'terrans' (people opposed to building super-AI).5

References

Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford. First published in 2014. Citations are from the pbk. edition, 2016. ISBN: 978-0198739838. Searches: https://www.amazon.com/s?k=978-0198739838 https://www.google.com/search?q=isbn+978-0198739838 https://lccn.loc.gov/2015956648

Butler, S. (1863). Darwin among the machines. The Press (Canterbury, New Zealand). Reprinted in Butler et al. (1923).

Butler, S., Jones, H., & Bartholomew, A. (1923). The Shrewsbury Edition of the Works of Samuel Butler Vol. 1. J. Cape. No ISBN. https://books.google.com/books?id=B-LQAAAAMAAJ Retrieved 27th Oct. 2020.

de Garis, H. (2005). The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines. ETC Publications. ISBN: 0882801546. Searches: https://www.amazon.com/s?k=0882801546 https://www.google.com/search?q=isbn+0882801546

Dyson, G. B. (1997). Darwin Among The Machines: The Evolution Of Global Intelligence. Basic Books. ISBN: 978-0465031627. Searches: https://www.amazon.com/s?k=978-0465031627 https://www.google.com/search?q=isbn+978-0465031627 https://lccn.loc.gov/2012943208

Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. Advances in Computers, 6, 31–88. https://exhibits.stanford.edu/feigenbaum/catalog/gz727rg3869 Retrieved 27th Oct. 2020.

Knight, W. (2018). Google just gave control over data center cooling to an AI. MIT Technology Review. 17th Aug. 2018. https://www.technologyreview.com/2018/08/17/140987/google-just-gave-control-over-data-center-cooling-to-an-ai/ Retrieved 26th Oct. 2020.

Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. ISBN: 978-0525558613. Searches: https://www.amazon.com/s?k=978-0525558613 https://www.google.com/search?q=isbn+978-0525558613 https://lccn.loc.gov/2019029688

Simon, H. A. (1996). The Sciences of the Artificial. MIT, 3rd ed. ISBN: 0262691914. Searches: https://www.amazon.com/s?k=0262691914 https://www.google.com/search?q=isbn+0262691914 https://lccn.loc.gov/96012633 Previous editions available at: https://archive.org/search.php?query=The%20sciences%20of%20the%20artificial

1Simon (1996) pp. 10-11, p. 162.

2Knight (2018)

3See, for example: Butler (1863); Good (1965); Dyson (1997); Bostrom (2014); Russell (2019).

4Bostrom (2014) p. 103, and note 27.

5de Garis (2005)