
Sign up to save your podcasts
Or


For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re9: They Can See YouRetraice1
On what is perceptible to AI, and AI controllers.
Air date: Saturday, 31st Oct. 2020, 12 : 10 PM Pacific/US.
AI, and the people who use it, can already see things that are invisible to most of us, including the future and any given individual's true colors. If we care about 'what's going on out there', we should care about 'seeing'.
1 Seeing Rods, cones and 4-D perceptionWhat comes after rods and cones?
Have you ever tried to imaging a four-dimensional cube? Maybe something that can perceive that?
Maybe that's what some crazy people are seeing when we think they're seeing crazy things.
Camouflage failureMaybe the next thing is what machines and AI, and the people who control them, are already doing: seeing us in ways we can't see each other, or ourselves, like our camouflage isn't working against them.2
See also Dyson on how AI would probably not reveal itself if it 'woke up'.3
Prediction machinesMaybe AI, or its controllers, are also seeing the future.4
2 'They'Is it the AI, or the controllers, or both, that can 'see'? We can argue about whether AI or machines are life-like, but the argument seems silly when we compare them to amoebas or fruit flies.
Networks change the estimation: most AIs are already networked. Are they one thing, or many? What about ourselves? Are we one or many?
Whoever controls AI is the 'they' who can see us.
They can see future-you—the Target debacleTarget, using Bayesian inference, knew a teenage girl was pregnant before her father did.5
Today and tomorrowWhatever 'they' can see today, they'll be able to see tomorrow, but in addition to new and more things.
3 Reality and fitnessSeeing reality might not be good for us, in terms of survival and reproduction.6
The fitness of the vertical illusionOn the vertical illusion, and its possible relation to fitness, see Jackson and Cormack.7
Minds connected and disconnectedSee Hoffman on how split brains, and connected brains, seem to change the number of conscious entities ('minds') in a given situation.8
The quantum chessboardReality, by the way, is a strange quantum world.9 And this is the world in which we and the machines, if they 'come to power', will be competing.
On empiricism and its approximation of reality, see Russell10 and Re1.11
4 So what? What's the work?It's not clear what to do about any of this. But the ideas need to be on the table.
5 CorrectionRe8: We said 'sir' Grey Walter, but he is not a knight, as far as we know.12
ReferencesAgrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Review Press. ISBN: 978-1633695672. Searches: https://www.amazon.com/s?k=978-1633695672 https://www.google.com/search?q=isbn+978-1633695672 https://lccn.loc.gov/2017049211
Anderson, R. (2015). He who pays the AI calls the tune. (pp. 201–203). In Brockman (2015).
Bell, J. S. (1987). Speakable and Unspeakable in Quantum Mechanics: Collected Papers on Quantum Philosophy. Cambridge, 2nd ed. ISBN: 0521523389. Searches: https://www.amazon.com/s?k=0521523389 https://www.google.com/search?q=isbn+0521523389 https://lccn.loc.gov/86032728
Brockman, J. (Ed.) (2015). What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence. Harper Perennial. ISBN: 978-0062425652. Searches: https://www.amazon.com/s?k=978-0062425652 https://www.google.com/search?q=isbn+978-0062425652 https://lccn.loc.gov/2016303054
Dyson, G. (2015). Analog, the revolution that dares not speak its name. (pp. 255–256). In Brockman (2015).
Ellenberg, J. (2014). How Not to Be Wrong: The Power of Mathematical Thinking. Penguin. ISBN: 978-0143127536. Searches: https://www.amazon.com/s?k=978-0143127536 https://www.google.com/search?q=isbn+978-0143127536 https://lccn.loc.gov/2014005394
Gefter, A., & Hoffman, D. (2016/04/25). The case against reality. The Atlantic. Previously published in Quanta. https://www.theatlantic.com/science/archive/2016/04/the-illusion-of-reality/479559/ Retrieved 31 Oct 2020.
Jackson, R. E., & Cormack, L. K. (2008). Evolved navigation theory and the environmental vertical illusion. Evolution and Human Behavior, 29, 299–304. https://liberalarts.utexas.edu/cps/_files/cormack-pdf/12Evolved_navigation_theory2009.pdf Retrieved 29th Oct. 2020.
Retraice (2020/09/07). Re1: Three Kinds of Intelligence. retraice.com. https://www.retraice.com/segments/re1 Retrieved 22nd Sep. 2020.
Retraice (2020/10/28). Re8: Strange Machines. retraice.com. https://www.retraice.com/segments/re8 Retrieved 29th Oct. 2020.
Russell, B. (1992). Human Knowledge: Its Scope and Limits. Routledge. First published in 1948. This edition 1992. ISBN: 0415083028. Different editions available at: https://archive.org/search.php?query=Human%20Knowledge%3A%20Its%20Scope%20and%20Limits
1https://www.retraice.com/retraice
2Anderson (2015) pp. 201-202.
3Dyson (2015) p. 256.
4Agrawal et al. (2018).
5Ellenberg (2014) p. 163.
6Gefter & Hoffman (2016/04/25).
7Jackson & Cormack (2008).
8Gefter & Hoffman (2016/04/25).
9Bell (1987) pp. 170-171.
10Russell (1992) p. 526.
11Retraice (2020/09/07)
12Retraice (2020/10/28).
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Ma8: Revolution and EvolutionMargin by Retraice
On before and after launch, and the six-sigma of Bill Smith. (With a philosophical and arithmetical digression in the PDF notes.)
Air date: Wednesday, 28th Oct. 2020, 5 : 30 PM Pacific/US.
1 Enjoy the revolutionBefore the launch of a business, or a particular product or service, there is time and space to do amazing things. After the launch, evolution and Darwin take over, which is a bit less fun.
The six-sigma of Bill SmithSix-sigma is 3.4 defects per million units. 'Sigma' is a statistical term that refers to standard deviations from a mean. If you produce 10 million units, a six-sigma process can produce essentially perfect quality in all but 34 of those 10 million units. The 34 units fall outside the sixth standard deviation from the mean unit, however you're measuring it. It's very close to perfect.
Smith's 1993 write-up of the idea is superb.1
DefectsA defect is 'a failure to satisfy a customer'—anyone from the end user to a co-worker.2
Measurements and experiencesA dissatisfaction—of any sort, of anybody involved—is a defect. If that's not vague enough, we can also imagine multiply dissatisfied customers, as in, 'I'm unhappy with more than one thing'. So our magic unit is … experiences? Two kinds in particular—satisfaction and dissatisfaction?
We're not counting dissatisfied customers or 'defective' products. We're counting experiences—of customers, colleagues, partners—everybody. If (dis)satisfaction experiences are easy to count when dealing with tangible things like parts and components, it is because there is something concrete in the world that corresponds to the subjective experience. If a piece is the wrong size, or a box has the wrong number of things in it, an unhappy customer almost has to report their experience in order to resolve the physical problem.3
But how do we count experiences that correspond, say, to a moment in an mp3 playback, or an impression of an image, which do not correspond to concrete objects and are therefore unlikely to be reported? It's not obvious.
We can't measure what a person thinks; we can only measure what a person does.
START: A philosophical and arithmetical digression on satisfaction, ratios and counting...Disclaimer: Within the below discussion there are attempts to explain—by untrained, non-professionals—simple, profound, widely misunderstood mathematical concepts such as quantity, ratio, and percent. We are neither mathematicians, nor teachers, nor practiced at this in any way other than by trial, error and reflection. The work below was done in response to a perceived need. Proceed with charity.
If you're quick and confident with arithmetic, or don't care for philosophy, skip this section. For the rest of us:
Why we're countingLet's assume we have a way of counting experiences. If we're able to count two kinds (satisfaction and dissatisfaction), what is the point of counting them? The point is to do our business in such a way that our quality level is 'six-sigma' good, i.e. that we have almost zero 'defects' (dissatisfaction experiences). 'Almost zero' = 3.4 dissatisfactions (or fewer) per 1 million experiences. (Really, we should say 'per 1 million satisfaction-or-dissatisfaction experiences connected to our product or service', since we shouldn't be counting (including in our scope of concern) experiences that are not connected, or are neither satisfying nor dissatisfying. But we'll take that for granted.)
'Ratio' is a word for something in the mindSince we're using the word 'per', we're in the mysterious realm of ratios, rates, fractions and division. There is a fuzziness and confusion in the world of mathematics about the difference between a ratio and a rate.4 You, yourself, will have to decide how to use these words, and how to translate them when you hear others use them. We've found no technical difference between the two concepts, only typical usage differences.
We recommend the following:
In the above, remember that 2∕1 = 2 because "2" is an abbreviation of "2/1"; the "/1" is there, it's just invisible.
If you're confused by percentages, try this: Because 'cent' is Latin for '100', always translate "percent" into "per cent" and then "per 100". So "6%" is "6 of something for every 100 of something".
And 2/1 = 200% because if I can make 2 widgets for every 1 widget that you make, then if you make 100 widgets, I'll have made 200, i.e. 200 per your 100, i.e. 200 per cent, 200 percent, 200%.
How the words 'ratio' and 'rate' are usedIf we look at typical uses, things become a bit more clear.
Paradigm uses of 'ratio' and 'rate':
The difference seems to be that ratios are comparing things that are fundamentally the same (humans, games), while rates are comparing things that are fundamentally different (miles vs. hours, dollars vs. years), but there are exceptions.
Expert opinionPractically:
"A ratio is a reckoning of the relationship of one thing to another. Mathematically speaking, it is the relationship gotten by dividing two things."6
"A rate generally involves a 'something else,' either two different kinds of units (such as distance per time), or just two distinct things measured with the same unit (such as interest money per loaned money)."7
Philosophically:
"[m]agnitudes of the same kind can be related to one another by ratios, and ratios can be compared with each other because they are relations perceived by our minds. In fact, the word for ratio, both in Greek and in Latin, is the same as the word for 'reason' or 'explanation' (logos in Greek, ratio in Latin). From the beginning, 'irrational' (alogos in Greek) could mean both 'without a ratio' and 'unreasonable.' "8
Aesthetically:
"The noun ratio is derived from the third principal part of the verb reor, reri, ratus, which means to think. If a line is divided at a point into two parts in such a way that the ratio of the larger part to the whole line is the same as the ratio of the smaller part to the larger part, then the common ratio is called the golden ratio because the rectangle whose base and altitude are the larger and smaller parts created by such a division is, according to those competent to have an opinion on such a matter, the most aesthetically pleasing of all rectangles that may be formed by dividing the given line and taking the parts to be the dimensions."9
Historically:
The roots of the word 'irrational', in both the mathematical and philosophical sense, reach all the way back to the ancient Greeks' difficulty comprehending , which is the length of the diagonal of a square with sides 1 unit long. The historian Carroll Quigley says that the Greeks' refusal10 to accept the fundamental irrationality of space (and therefore reality), as demonstrated by , is what killed ancient science because it denigrated observation, testing and experiment.11
Also, the mathematics teacher Steven Schwartzman says the word ratio is deeply connected to the word read:
"The Indo-European root underlying Latin ratio is plausibly ar- 'to fit together,' so that the Latin ratio developed from the idea of fitting numbers together in the sense of comparing them. Related borrowings include arthritis, a disease of the joints (where bones are fitted together), and adorn (to fit things together in a decorative way). Also related is native English read, to fit words together to make sense out of them."12
More practically, a ratio is:
"[a] comparison of two like quantities by dividing one by the other."13 [emphasis added]
Back to businessThe two quantities we're comparing are like because they're both experiences: satisfaction(s) and dissatisfaction(s). If we chose a different kind of thing, such as 'customer complaints' or 'broken widgets' we might obscure what we care about. If a widget is broken and it doesn't bother the customer, it's not a defect; if a customer is dissatisfied but doesn't complain, it is a defect.
Number sizesWe're going to be dealing very large numbers, like 1,000,000 and 10,000,000, and very small numbers, like 34 and 3.4 and 0.0000034. Why? Because we want to do a lot of business and waste very little of it. If we produce 10 experiences and 1 is dissatisfying to a customer, we've wasted 10% of our time and energy.14 Wouldn't it be nice to waste nothing, and convert all of our work into good customer experiences (and therefore revenue and profit)? Well, 3.4 in 1,000,000 is very close to 'nothing', so that's our aim, and those are the sorts of numbers we have to handle.
Drawing the invisibleWe're going to try to represent these big and small numbers, and the mysterious relationship between them (the ratio), with text on a page. But it's crucial to remember that numbers are invisible and mysterious and they float around and do not care whether someone tries to draw them. The exact same ghostly number can be drawn in different ways, as long as we all agree on the rules of drawing. We can agree that the drawing '1' is a drawing of the number we call 'one', and we can also agree that the drawings '1/1' and '2/2' are also drawings of that same invisible number. Always remember that the number is not on the page, because it's invisible; only the drawings of it can be on the page, and there are many different drawings of a given number. The same applies to ratios and other mathematical ghosts.
Statements about the invisibleSo lets look at the ratio of 3.4 dissatisfaction-experiences per 1 million total experiences. (The use of a colon ":" between the numbers is an abbreviation of the ÷ symbol.)15
This is a mathematical statement (hence the = sign). The statement says: 'These two drawings are of the same ratio.'16 Alternatively, it says 'These two mathematical expressions represent the same mysterious, invisible thing'.
In more familiar notation:
Or:All three drawings are of the same, mysterious, invisible thing.
Thinking about the invisibleSo, our magic number is 0.0000034? 0.0000034 what? 0.0000034 dissatisfactions per 1 experience? What does that mean? Let's put it back in original form: 3.4 dissatisfactions per 1,000,000 experiences. How can we have 3 + 0.4 dissatisfactions? It's still not clear.
What if we think in percentages? Does that help? We can convert this magic number, or ratio, into a percentage ('per one hundred'): if our magic ratio can be expressed as "0.0000034 per 1" then it can also be expressed as "0.00034 per 100", or "0.00034%".
So we want a 0.00034% defect ratio (or lower). What does that mean? Let's try notation again:
This says '3.4 one-millionths is the same mysterious number as 34 ten-millionths'. I.e., if we can only have 3.4 dissatisfactions in a million, we can only have 34 dissatisfactions in 10 million.
autoIn more generous notation, it looks like this:
This equation says 'the left mysterious ratio relationship is the same as the right mysterious ratio relationship'.
We've added the parentheses to remind you that we're talking about a ratio, not the two individual numbers and the division operation that are the components of it. It's like we're talking about a box, and you can see inside the box, but we're talking about the box as a whole. In this case, we're saying these two boxes are the same on the outside, even though they're different on the inside. But this kind of box, with these exact dimensions on the outside, is what we care about. This is six-sigma quality.
The 3.4-per-1,000,000 ratio is the same as approximately 1 defect per 300,000 experiences, because:
So if we want 34 or fewer dissatisfactions per 10,000,000 experiences, we can also say that we want approximately 1 dissatisfaction (or fewer) for every 300,000 experiences. Calculating our dissatisfaction ratioTo calculate our dissatisfaction ratio (or 'defect' 'rate'), we first get the total number of (relevant) experiences:
autoWe then want to put the number of experiencesdissatisfaction over the number of experiencestotal and see if the equivalent number (or ratio) is higher or lower than 0.0000034, aka 0.00034%, aka 3.4 per 1 million, aka 34 per 10 million, aka (approximately) 1 dissatisfaction per 300,000 experiences.
The other invisible thingThe real problem is counting the experiences. Remember, it's not the units of product, or customer complaints, or anything else that's easy to measure. We're interested in the experiencesdissatisfaction, a quantity of phenomena in the world that are hard to observe, and therefore hard to count.
If we're manufacturing, say, a radio, then we know how many of them don't pass final inspection, or get returned by customers as defective. But those are just the most obvious experiencesdissatisfaction. What about dissatisfaction within oneself, or of a colleague? What about multiple dissatisfactions in a single moment? Or with a single unit? The key is in the counting, and in counting experiences, we will have to reflect further on our businesses.
...END digression Six-sigma literature is mostly garbageAcronyms and jargon are often the antithesis of clear thinking and expression, as six-sigma books and articles tend to prove.
Any dissatisfaction, any 'customer'By Smith's and Motorola's definition, any experience of dissatisfaction on the part of any person involved in the business is a defect.17
However, faults that go unnoticed by the company are unlikely to be noticed by the customer.18
Revolution is before productionRevolutionary changes are only practical on paper; once production starts, evolution takes over.19
There's nothing revolutionary coming out of Google or Apple or Samsung or Toyota anymore. We can say that Tesla was doing revolutionary things recently, but only before they started production.20
Darwin's territoryRevolution is bloody and merciless, but you probably have to at least like it (if not love it the way you should love the revolutionary phase) to do well in business.
Precision, tolerance, and simplicityYou've either gotta sharpen your aim, or pick a bigger target, to improve your quality.
Here's what's good:
Alternatively: checklists, fail-safes and simplicity.
A defect at Retraice, Inc.We put up an incorrect show image, and the mistake was totally predictable and unnecessary.
Fixing is more expensive than preventing.21
We have many checks in place, even though we're a simple podcasting company, because we're trying to be very high quality. Most podcasts are not. Compare Youtube—how many hours are uploaded every day, and how many are actually watched?
ChecklistsSee Gawande on the importance and power of checklists, especially in aircraft and hospitals.22 On the nature of improving, see also Gawande.23
Don't be a wantrepreneurEnjoy it, but don't stay in the revolutionary phase for too long.
ReferencesGawande, A. (2009). The Checklist Manifesto: How to Get Things Right. Henry Holt and Co., Kindle ed. ISBN: 978-0805091748. Searches: https://www.amazon.com/s?k=978-0805091748 https://www.google.com/search?q=isbn+978-0805091748 https://lccn.loc.gov/2009046888
Gawande, A. (2011). Better: A Surgeon's Notes on Performance. Henry Holt and Co., Kindle ed. ISBN: 978-0805082111. Searches: https://www.amazon.com/s?k=978-0805082111 https://www.google.com/search?q=isbn+978-0805082111 https://lccn.loc.gov/2006046962
Gouvêa, F. Q. (2008). From numbers to number systems. (pp. 77–83). In Gowers (2008).
Gowers, T. (Ed.) (2008). The Princeton Companion to Mathematics. Princeton University Press. ISBN: 978-0691118802. Searches: https://www.amazon.com/s?k=978-0691118802 https://www.google.com/search?q=isbn+978-0691118802 https://lccn.loc.gov/2008020450
Klaf, A. A. (1964). Arithmetic Refresher. Dover. ISBN: 0486212416. Searches: https://www.amazon.com/s?k=0486212416 https://www.google.com/search?q=isbn+0486212416 https://lccn.loc.gov/64018856
Liker, J. (2004). The Toyota Way: 14 Management Principles from the World's Greatest Manufacturer. McGraw-Hill. ISBN: 0071392319. Searches: https://www.amazon.com/s?k=0071392319 https://www.google.com/search?q=isbn+0071392319 https://lccn.loc.gov/2004300007
Lo Bello, A. (2013). Origins of Mathematical Words: A Comprehensive Dictionary of Latin, Greek, and Arabic Roots. Johns Hopkins University Press. ISBN: 978-1421410982. Searches: https://www.amazon.com/s?k=978-1421410982 https://www.google.com/search?q=isbn+978-1421410982 https://lccn.loc.gov/2013005022
Peterson (2001/10/23). Rate vs. ratio. mathforum.org. Author name given only as 'Doctor Peterson'; we were unable to find a first name. http://mathforum.org/library/drmath/view/58042.html Retrieved 30th Oct. 2020.
Quigley, C. (1961). The Evolution of Civilizations. Macmillan (reprinted by Liberty Fund 1979). ISBN: 0913966576. Searches: https://www.amazon.com/s?k=0913966576 https://www.google.com/search?q=isbn+0913966576 https://lccn.loc.gov/79004091
Russell, B. (1938). The Principles of Mathematics. W. W. Norton & Company, 2nd ed. ISBN: 0393314049. First published in 1903, 2nd ed. with new introduction 1938. Citations are from 1996 reprint. Searches: https://www.amazon.com/s?k=0393314049 https://www.google.com/search?q=isbn+0393314049 https://lccn.loc.gov/38027192
Schwartzman, S. (1994). The Words of Mathematics: An Etymological Dictionary of Mathematical Terms Used in English. The Mathematical Association of America. ISBN: 0883855119. Searches: https://www.amazon.com/s?k=0883855119 https://www.google.com/search?q=isbn+0883855119 https://lccn.loc.gov/93080612
Smith, B. (1993a). Six-sigma design. IEEE Spectrum, 30(9), 43–47. September 1993. https://www.scribd.com/document/419712315/Six-sigma-Design-Quality-Control Retrieved ca. 1st Feb. 2020.
Smith, B. (1993b). Total customer satisfaction as a business strategy. Quality and Reliability Engineering International, 9(1), 49–53. https://onlinelibrary.wiley.com/doi/abs/10.1002/qre.4680090109 Retrieved ca. 1st Feb. 2020.
Index Apple, 5Google, 5
Samsung, 5
Tesla, 5 Toyota, 5
Youtube, 5
1Smith (1993b). See also Smith (1993a).
2Smith (1993a) p. 43.
3Of course, we can imagine a customer dissatisfied with the number of cup-holders in a car, a physical problem that yet probably won't be reported back to Toyota. Side-note: see Liker (2004) pp. 229-230 on the engineer who drove a Toyota minivan all over the U.S. in order to find design improvements, one of which was increasing the number of cup-holders.
4See Peterson (2001/10/23) for a thoughtful discussion.
5A quantity, let's say, can be either a multitude or a magnitude. A multitude is countable, discrete, like apples. A magnitude is fluid, continuous, like string. We don't have to imagine breaking up each apple into an apple, but we do have to imagine breaking up a string into centimeters. Using these terms this way is a choice, not a fact. These words correspond to deep, complicated ideas. Cf. Russell (1938) p. 159 ff.
6Schwartzman (1994) p. 183.
7Peterson (2001/10/23)
8Gouvêa (2008) p. 79.
9Lo Bello (2013) p. 270.
10For example: Socrates, Plato, and to some extent Aristotle.
11Quigley (1961) pp. 90-91.
12Schwartzman (1994) p. 183.
13Klaf (1964) p. 159.
14We'll set aside the value of learning from mistakes and experience, for the purpose of simplicity.
15Klaf (1964) p. 160.
16A statement that two ratios are equal is called a proportion—Klaf (1964) p. 168. Although, the roots of the word are in Latin and mean 'with respect to one's share'—Lo Bello (2013) p. 260.
17Smith (1993a) p. 43.
18Smith (1993a) p. 44.
19Smith (1993a) p. 47.
20Cf. Quigley on how outward-looking, purposeful social instruments tend to become, over time, inward-looking, vested social institutions. Quigley (1961) p. 101-103.
21During the livestream, we said 'kanban' cord, but the Toyota term for the stop-assembly-line cord is 'andon', and it's more often a button. Liker (2004) p. 23 and p. 130. This correction will be mentioned in Ma9.
22Gawande (2009).
23Gawande (2011) p. 2.
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re8: Strange MachinesA survey of the idea that technology is creatures.
Air date: Wednesday, 28th Oct. 2020, 3 : 30 PM Pacific/US.
1 The Problem We should call them something elseIf machines or artifacts have goals, we should call them creatures, or something else.
High-altitude fruitThis is just a sampling of the available ideas.
We focus on the books and papers because there's plenty of low-hanging fruit already available in electronic media—movies, documentaries, radio, TV, etc. It's hard to add homework to the already-heavy load of electronic production, so these sources don't tend to make the cut.
Simon—the rules are the sameWe can look at living systems the way we look at artifacts—as interfaces between inner- and outer-environments.1
Grey Walter's tortoisesAlong with biologically engineered organisms, some robots2 might be examples of life forms that did not, strictly speaking, evolve organically.3
Butler—war to the deathOur interests are inseparable from machines, but they're on the verge of reproduction, so we should destroy them.4
Dyson—they're not imaginarySmart computer programs that seem like creatures are very real.5
Wolfram's simple programsSimple cellular automata programs produce wild complexity.6
Yudkowsky on fire alarmsFire alarms make it socially acceptable to react to a situation that might be dangerous but not obvious. There will be no equivalent mechanism for dangerous AI.7
I. J. Good—take science fiction seriouslyIf machines can do what humans do, they can improve machines, which is themselves, which means boom.8
'unquestionably'If the argument focuses on whether it is certain that an intelligence explosion will happen, we're probably missing the point.9
Yudkowsky—smartish stuffWe shouldn't argue too much about whether something is truly intelligent; if it accomplishes dramatic things, our definitions don't matter.10
See Legg and Hutter for more technical work on the definition(s) of intelligence.11
S. Russell and Norvig—operating on their ownAI and robotics are unlike other dangerous technologies because of autonomy; we need to engineer safety, not hope for it.12
Two meanings of 'the singularity'John von Neumann gives the 'unpredictable' definition, as recollected and paraphrased by Stanislaw Ulam:
"Quite aware that the criteria of value in mathematical work are, to some extent, purely aesthetic, [von Neumann] once expressed an apprehension that the values put on abstract scientific achievement in our present civilization might diminish: 'The interests of humanity may change, the present curiosities in science may cease, and entirely different things may occupy the human mind in the future.' One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue."13
Ray Kurzweil gives the upgrade-or-bust definition, in dialogue form.
MOLLY 2004: … But seriously, I'm having trouble keeping up with all of this stuff flying at me as it is. How am I going to keep up with the pace of the Singularity? …
GEORGE 2048: … You'll be able to grasp what I'm up to if that's what you really want.
MOLLY 2004: What, by becoming…
RAY: Enhanced?
MOLLY 2004: Yes, that's what I was trying to say.
GEORGE 2048: Well, if our relationship is to be all that it can be, then it's not a bad idea."14
A moral challengeIf we take for granted that human life and the human species are worth preserving we have a moral disagreement with those who accept the prospect of machines inheriting the Earth.15
S. Russell—the user's mindThe environment modified by content-selection algorithms (e.g. those used in social media) is the user's mind. The goal is to make the mind more predictable.16
Dyson—worry less about intelligenceWe should worry more about self-reproduction, communication and (analog) control than machine intelligence.17
Smallberg—energy sources and replicationWhen superintelligent machines 'start replicating and looking for an energy source solely under their control', will have crossed a threshold.18
A digression on searchPerhaps they've crossed a threshold simply by looking for anything. Is 'looking for' the same as searching, or scanning?19
Dietterich—reproduction with autonomyIf there are four steps to an intelligence explosion—machines conducting experiments, discovering new structures, building mechanisms to exploit discoveries, and granting autonomy and resources to the new mechanisms themselves—it's the fourth step that's dangerous.20
2 The workWhile these ideas are troubling and fascinating and fun21 to think about, what to do?
Bostrom—deferred gratificationWe should stop work on high-minded, laudable long-term goals, for now, in favor of focusing on surviving the advent of superintelligence.22
Our civilization is evidence of capacityThe spectacular achievements of our civilization(s) are a hint that we can achieve the necessary spectacular requirements of AI safety (assuming the nature of the threat).
Skyscrapers seem taller than they areOn how tall things seem longer than far things, see Jackson and Cormack.23
ReferencesBostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford. First published in 2014. Citations are from the pbk. edition, 2016. ISBN: 978-0198739838. Searches: https://www.amazon.com/s?k=978-0198739838 https://www.google.com/search?q=isbn+978-0198739838 https://lccn.loc.gov/2015956648
Brockman, J. (Ed.) (2015). What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence. Harper Perennial. ISBN: 978-0062425652. Searches: https://www.amazon.com/s?k=978-0062425652 https://www.google.com/search?q=isbn+978-0062425652 https://lccn.loc.gov/2016303054
Brockman, J. (Ed.) (2019). Possible Minds: Twenty-Five Ways of Looking at AI. Penguin. ISBN: 978-0525557999. Searches: https://www.amazon.com/s?k=978-0525557999 https://www.google.com/search?q=isbn+978-0525557999 https://lccn.loc.gov/2018032888
Butler, S. (1863). Darwin among the machines. The Press (Canterbury, New Zealand). Reprinted in Butler et al. (1923).
Butler, S., Jones, H., & Bartholomew, A. (1923). The Shrewsbury Edition of the Works of Samuel Butler Vol. 1. J. Cape. No ISBN. https://books.google.com/books?id=B-LQAAAAMAAJ Retrieved 27th Oct. 2020.
de Garis, H. (2005). The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines. ETC Publications. ISBN: 0882801546. Searches: https://www.amazon.com/s?k=0882801546 https://www.google.com/search?q=isbn+0882801546
Dietterich, T. G. (2015). How to prevent an intelligence explosion. (pp. 380–383). In Brockman (2015).
Dyson, G. (2019). The third law. (pp. 31–40). In Brockman (2019).
Dyson, G. B. (1997). Darwin Among The Machines: The Evolution Of Global Intelligence. Basic Books. ISBN: 978-0465031627. Searches: https://www.amazon.com/s?k=978-0465031627 https://www.google.com/search?q=isbn+978-0465031627 https://lccn.loc.gov/2012943208
Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. Advances in Computers, 6, 31–88. https://exhibits.stanford.edu/feigenbaum/catalog/gz727rg3869 Retrieved 27th Oct. 2020.
Harris, S. (2016). Can we build AI without losing control over it? — Sam Harris. TED. https://youtu.be/8nt3edWLgIg Retrieved 28th Oct. 2020.
Holland, O. (2003). Exploration and high adventure: the legacy of Grey Walter. Phil. Trans. R. Soc. Lond. A, 361, 2085–2121. https://www.researchgate.net/publication/9025611 Retrieved 22nd Nov. 2019. See also: https://www.youtube.com/results?search_query=grey+walter+tortoise+
Jackson, R. E., & Cormack, L. K. (2008). Evolved navigation theory and the environmental vertical illusion. Evolution and Human Behavior, 29, 299–304. https://liberalarts.utexas.edu/cps/_files/cormack-pdf/12Evolved_navigation_theory2009.pdf Retrieved 29th Oct. 2020.
Kurzweil, R. (2005). The Singularity Is Near: When Humans Transcend Biology. Penguin. ISBN: 978-0143037880. Searches: https://www.amazon.com/s?k=978-0143037880 https://www.google.com/search?q=isbn+978-0143037880 https://lccn.loc.gov/2004061231
Legg, S., & Hutter, M. (2007a). A collection of definitions of intelligence. Frontiers in Artificial Intelligence and Applications, 157, 17–24. June 2007. https://arxiv.org/abs/0706.3639 Retrieved ca. 10 Mar. 2019.
Legg, S., & Hutter, M. (2007b). Universal intelligence: A definition of machine intelligence. Minds & Machines, 17(4), 391–444. December 2007. https://arxiv.org/abs/0712.3329 Retrieved ca. 10 Mar. 2019.
Retraice (2020/09/07). Re1: Three Kinds of Intelligence. retraice.com. https://www.retraice.com/segments/re1 Retrieved 22nd Sep. 2020.
Retraice (2020/09/08). Re2: Tell the People, Tell Foes. retraice.com. https://www.retraice.com/segments/re2 Retrieved 22nd Sep. 2020.
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. ISBN: 978-0525558613. Searches: https://www.amazon.com/s?k=978-0525558613 https://www.google.com/search?q=isbn+978-0525558613 https://lccn.loc.gov/2019029688
Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches: https://www.amazon.com/s?k=978-0134610993 https://www.google.com/search?q=isbn+978-0134610993 https://lccn.loc.gov/2019047498
Simon, H. A. (1996). The Sciences of the Artificial. MIT, 3rd ed. ISBN: 0262691914. Searches: https://www.amazon.com/s?k=0262691914 https://www.google.com/search?q=isbn+0262691914 https://lccn.loc.gov/96012633 Previous editions available at: https://archive.org/search.php?query=The%20sciences%20of%20the%20artificial
Smallberg, G. (2015). No shared theory of mind. (pp. 297–299). In Brockman (2015).
Ulam, S. (1958). John von Neumann 1903-1957. Bull. Amer. Math. Soc., 64, 1–49. https://doi.org/10.1090/S0002-9904-1958-10189-5 Retrieved 29th Oct. 2020.
Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman and Company. ISBN: 0716704633. Also available at: https://archive.org/details/computerpowerhum0000weiz
Wolfram, S. (Ed.) (2002). A New Kind of Science. Wolfram Media, Inc. ISBN: 1579550088. Searches: https://www.amazon.com/s?k=1579550088 https://www.google.com/search?q=isbn+1579550088 https://lccn.loc.gov/2001046603
Yudkowsky, E. (2013). Intelligence explosion microeconomics. Machine Intelligence Research Institute. Technical report 2013-1. https://intelligence.org/files/IEM.pdf Retrieved ca. 9th Dec. 2018.
Yudkowsky, E. (2017). There's no fire alarm for artificial general intelligence. Machine Intelligence Research Institute. 13th Oct. 2017. https://intelligence.org/2017/10/13/fire-alarm/ Retrieved 9th Dec. 2018.
1Simon (1996) p. 6.
2During the livestream, Walter was referred to as 'Sir' Walter. This was an error; Walter was not knighted, as far as we're aware.
3Holland (2003) pp. 2104-2105. Holland is skeptical of the self-awareness interpretation of the tortoises' behavior.
4Butler (1863) pp. 184-185
5Dyson (1997) p. xii.
6Wolfram (2002) e.g. p. 30.
7Yudkowsky (2017).
8Good (1965) p. 33.
9See Yudkowsky (2013) p. 1 for sources on the discussion.
10Yudkowsky (2013) p. 9.
11Legg & Hutter (2007b), Legg & Hutter (2007a).
12Russell & Norvig (2020) p. 1001
13Ulam (1958) p. 5.
14Kurzweil (2005) p. 31.
15Russell & Norvig (2020) p. 1005. See also de Garis (2005) on 'cosmists', p. 81 ff.
16Russell (2019) pp. 8-9.
17Dyson (2019) p. 40. On control, see also Retraice (2020/09/08) p. 3, and Weizenbaum (1976) pp. 124-126.
18Smallberg (2015) p. 299.
19See Retraice (2020/09/07) p. 1 on a loose definition of artificial intelligence: "whatever it is that makes certain machines and computers seem to know things, and to act like they know things." [emphasis added]
20Dietterich (2015) p. 382
21On the 'fun' of death by science fiction, see Harris (2016) ca. 1:00 min ff.
22Bostrom (2014) p. 315.
23Jackson & Cormack (2008), especially p. 301 ('results').
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Ma7: Reading and Writing Margin by RetraiceOn detecting, navigating and creating reality in business.
Air date: Monday, 26th Oct. 2020, 6 : 40 PM Pacific/US.
1 ReadingOn the one hand, we're trying to see clearly what is already the case.
Jobs on reading what's not yet writtenThe Jobs quote:
"Some people say, 'Give the customers what they want.' But that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, 'If I'd asked customers what they wanted, they would have told me, 'A faster horse!' ' People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page."1
DetectionHe didn't just say he shows people what they want before they know they want it. He said he tries to read (figure out) what they want before others (customers, competitors) do. If he's reading, he's detecting something that's already there.
Ries says something similar.2
"What do people want?" "What's the version of what I'm doing that people want?" These are attempts to detect something.
SteeringWhat do you do once you detect something? You steer, navigate, move around and avoid obstacles and pursue promise. auto
True northThis is an idea that Ries and Doerr talk about, a sort of guiding ethos or mission.3 Ries equates it with his use of the term 'vision'.
Effecting is not detectingEvery move you make, or don't make, is affecting things. And effecting (change) is not detecting.
2 Writing You are writing the pageYou might not be doing exactly what you set out to do, but either way, you're writing.
An alternative statement to Jobs' (?): "Customers don't want things until we cause them to want them."
You're changing the future—not fundamentally differently than anyone does, but you are changing it.
You're changing what's around youWithin the boundaries of nature is the stuff you care about, which you are changing. Bezos said something like: If we focus on what's not going to change, that's the sort of stuff that enables us to see around corners.4
We are in business to writeYes, we want to read what's not yet written, and see around corners. But we are in business to write on the page, to build around the corner.
3 So What?It's a terrible, invaluable question. It feels bad to not have a good answer.
Wrote-the-book is better than read-the-book.
When there are no widgetsWe've conveyed an idea today, and if we take for granted that it was valid and successful, still we do not know whether it was productive. So what? There are no widgets. A stream, or an mp3, is not a widget, is it?
A question for every businessIf you succeed, so what?
ReferencesDoerr, J. (2017). Measure What Matters: OKRs: The Simple Idea that Drives 10x Growth. Penguin Random House. ISBN: 978-0241348482. Searches: https://www.amazon.com/s?k=978-0241348482 https://www.google.com/search?q=isbn+978-0241348482 https://lccn.loc.gov/2018002727
Isaacson, W. (2011). Steve Jobs. Simon & Schuster. ISBN: 978-1451648539. Searches: https://www.amazon.com/s?k=978-1451648539 https://www.google.com/search?q=isbn+978-1451648539 https://lccn.loc.gov/2011045006
Ries, E. (2011). The Lean Startup. Currency / Crown / Penguin Random House. ISBN: 978-0307887894. Searches: https://www.amazon.com/s?k=978-0307887894 https://www.google.com/search?q=isbn+978-0307887894 https://lccn.loc.gov/2011012100
Index Bezos, Jeff, 1Henry Ford, 1
1Isaacson (2011) p. 567
2Ries (2011) p. 10 and throughout.
3Ries (2011) p. 22, Doerr (2017) p. 104.
4It was a video (YouTube) of a stage interview with Bezos. Write to us if you have the link.
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re7: Artifactual GoalsOn goals producing artifacts-producing-goals, and machines coming alive.
Air date: Monday, 26th Oct. 2020, 5 : 50 PM Pacific/US.
1 Goals produce artifactsSimon says something like this, and that goals produce more goals.1
Artifacts might absorb goalsIt really depends on the physical interpretation of 'goal': does an idea, or intention, leave the mind, or become somehow embodied or instantiated or imitated in, say, an abacus?
Artifacts cause goalsA constructed abacus often causes the goal of displaying that abacus.
Artifacts might absorb their own goalsAny machines that move, that have moving parts, have something like goal absorption and exsorption happening between the parts inside them.
Artifacts might absorb each others' goalsUsing two abaci together, might a goal move from one to the other via you?
Machines aliveThe animated behavior of machines suggests a question: At what point do 'inanimate' things start having their own goals? Consider AlphaZero, managing Google data center power supplies.2
Consider SCADA systems, or nuclear missile silos, which have so much automation (animation) built into them.
Goals coming out of machinesThis is a well-worn topic.3
The goal is the key. If there is something about goals that we don't understand, that fact might be good news or bad news about the future of AI (and humanity).
Artifacts exchanging goalsIf goals can move between entities of any sort, they might move between entities of all sorts.
2 What would we do if we knew?If we knew whether machines can have goals, or share them, or become lifelike, or become dangerous, what action, on our part, would that imply? It's not obvious.
How could the world be reorganized to prevent dangerous super AI?
Is it something we're doing, or something that's happening to us?
TotalitarianismTotalitarianism is one of the things Nick Bostrom considers.4
WarHugo de Garis predicts an 'artilect war' between 'cosmists' (people bent on building super-AI, no matter what the consequences) and 'terrans' (people opposed to building super-AI).5
ReferencesBostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford. First published in 2014. Citations are from the pbk. edition, 2016. ISBN: 978-0198739838. Searches: https://www.amazon.com/s?k=978-0198739838 https://www.google.com/search?q=isbn+978-0198739838 https://lccn.loc.gov/2015956648
Butler, S. (1863). Darwin among the machines. The Press (Canterbury, New Zealand). Reprinted in Butler et al. (1923).
Butler, S., Jones, H., & Bartholomew, A. (1923). The Shrewsbury Edition of the Works of Samuel Butler Vol. 1. J. Cape. No ISBN. https://books.google.com/books?id=B-LQAAAAMAAJ Retrieved 27th Oct. 2020.
de Garis, H. (2005). The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines. ETC Publications. ISBN: 0882801546. Searches: https://www.amazon.com/s?k=0882801546 https://www.google.com/search?q=isbn+0882801546
Dyson, G. B. (1997). Darwin Among The Machines: The Evolution Of Global Intelligence. Basic Books. ISBN: 978-0465031627. Searches: https://www.amazon.com/s?k=978-0465031627 https://www.google.com/search?q=isbn+978-0465031627 https://lccn.loc.gov/2012943208
Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. Advances in Computers, 6, 31–88. https://exhibits.stanford.edu/feigenbaum/catalog/gz727rg3869 Retrieved 27th Oct. 2020.
Knight, W. (2018). Google just gave control over data center cooling to an AI. MIT Technology Review. 17th Aug. 2018. https://www.technologyreview.com/2018/08/17/140987/google-just-gave-control-over-data-center-cooling-to-an-ai/ Retrieved 26th Oct. 2020.
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. ISBN: 978-0525558613. Searches: https://www.amazon.com/s?k=978-0525558613 https://www.google.com/search?q=isbn+978-0525558613 https://lccn.loc.gov/2019029688
Simon, H. A. (1996). The Sciences of the Artificial. MIT, 3rd ed. ISBN: 0262691914. Searches: https://www.amazon.com/s?k=0262691914 https://www.google.com/search?q=isbn+0262691914 https://lccn.loc.gov/96012633 Previous editions available at: https://archive.org/search.php?query=The%20sciences%20of%20the%20artificial
1Simon (1996) pp. 10-11, p. 162.
2Knight (2018)
3See, for example: Butler (1863); Good (1965); Dyson (1997); Bostrom (2014); Russell (2019).
4Bostrom (2014) p. 103, and note 27.
5de Garis (2005)
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Ma6: The Back FootMargin by Retraice
On waiting, being on defense and getting blindsided.
Air date: Sunday, 25th Oct. 2020, 08 : 20 PM Pacific/US.
1 Sundays and MondaysPeople are busy on Mondays, and they'll tend to make you busy, and so waiting to start the week puts you at a disadvantage.
Time zones mean it's easy to start the week behind, or be compelled to work into the weekend.
WaitingWaiting can be a mistake, and a bad habit.
Being on the defensiveCathy Hughes1 made the point that, when she was unable to pay her bills, she proactively warned and negotiated with her creditors.
Tipping people overBeing pushed backward off balance can turn even an armored soldier into something like a flipped turtle.
Being blindsidedAfter the blindside hit, you're on the defensive. But before it, you're in a strange place of ignorance.
Andy Grove:
"If you work in one of these industries and you are in middle management, you may very well sense the shifting winds on your face before the company as a whole and sometimes before your senior management does. Middle managers—especially those who deal with the outside world, like people in sales—are often the first to realize that what worked before doesn't quite work anymore; that the rules are changing."2
You're the CEO and the janitorWhich means you're also middle management, which means you might 'sense the shifting winds', and avoid being blindsided.
Cathy Hughes said the only thing she didn't know how to do at her radio station was fire up the transmitter.3
War time and peace time CEOsBen Horowitz wrote that there are two kinds of CEO, and that you have to know which you are, or which kind you need.4 But isn't it better to win the war without fighting?
Stay off the back foot.
ReferencesGrove, A. S. (1996). Only the Paranoid Survive: How to Exploit the Crisis Points That Challenge Every Company. Currency / Doubleday. ISBN: 0385483821. Searches: https://www.amazon.com/s?k=0385483821 https://www.google.com/search?q=isbn+0385483821 https://lccn.loc.gov/96013509
Horowitz, B. (2014). The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers. Harper Business. ISBN: 978-0062273208. Searches: https://www.amazon.com/s?k=978-0062273208 https://www.google.com/search?q=isbn+978-0062273208 https://lccn.loc.gov/2017448298
Hughes, C., & Raz, G. (2017). Radio One: Cathy Hughes. NPR's How I Built This with Guy Raz. 14th Aug. 2017. https://www.npr.org/2017/09/29/542650845/radio-one-cathy-hughes Retrieved 23rd Oct. 2020.
1Hughes & Raz (2017)
2Grove (1996) p. 21.
3Hughes & Raz (2017)
4Horowitz (2014) p. 224 ff.
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re6: InterfaceOn what AI is or isn't, and whether it's between things.
Air date: Sunday, 25th Oct. 2020, 07 : 20 PM Pacific/US.
1 No one knows what AI isSuppose we did know. We should be able to answer at least one of the following questions.
Six questions ∙ Who is AI? This doesn't feel like a question we can ask yet. ∙ Where is AI? In the computers? In a robot? Where is the AI in a super computer AI warehouse? It could be in the bare metal. But if the power is off, is that still AI? Maybe it's in the electricity. But electricity is hard to think about.1 Maybe it's in the patterns of something, as revealed by the passage of time. But where, specifically, is the AI? Is it the whole pattern? Part of it? ∙ When is AI? When is anything? When is now? We need atomic clocks and a philosophy of experience to do serious work on 'when' questions. ∙ How is AI? That's a weird question. We'll skip it for now. ∙ Why is AI? This leads to a chain reaction: 'Because somebody built it.' Why did they build it? etc. ∙ Which is AI? All the other 'W' questions can be interpreted as 'which' questions, since they can be seen as selecting from a group. Which is AI?In Ma4: Assumptions (A Wrestling Match)2 I said that the big AI stuff will make you question your cup of coffee. Here we go.
AI is an oceanThoughts about AI are widespread. Movies about AI are familiar.
There is an ocean of stuff going on out there—strategic intelligence, artificial intelligence, natural intelligence. (Remember, Retraice is about 'what's going on out there'.)
2 Herbert Simon on 'the artificial'Simon, along with Allen Newell, wrote significant early AI programs. Simon also won the Nobel prize in economics.3
An interfaceSide note: the term 'artificial intelligence' was coined by John McCarthy in 1955.4 We can distinguish artifact and artifice, but he probably didn't.
Simon says artifacts are like interfaces.
"An artifact can be thought of as a meeting point—an 'interface' in today's terms—between an 'inner' environment, the substance and organization of the artifact itself, and an 'outer' environment, the surrounds in which it operates. If the inner environment is appropriate to the outer environment, or vice versa, the artifact will serve its intended purpose."5
ArtifactsArtifacts include: a watch, a computer, a keyboard, a highlighter, a cup of coffee, a sticky note.
A batteryInside, its a chemistry experiment. Outside, it is two leads, and a label with color and style. It's put together in just the right way so that we get power out of it, if we use it in just the right way.
Goals and environmentsPerhaps artifacts are interfaces between goals and environments.
Where's the goal? When is it? Which goal?
Goals connected to batteriesThink of the many people—and goals—involved in the production of a battery. Do the goals branch off from the path, the destiny of the battery? Or do they accrue to it? Are they drawn to it in some sense?
Where is a goal? Where is a computer program? Where is a thought? (Where is AI?)
Simple and complex artifactsA battery is simple. Artificial intelligence is either complicated or complex. But they're both artifacts.
3 Amendments, correctionsI said that Bruce Schneier was a physicist, but he only studied physics.
We found the Dulles quote about conspiracy.6
Russell said something like what O'Shea said, but in 1921.7
ReferencesAmdahl, K. (1991). There Are No Electrons: Electronics for Earthlings. Clearwater Publishing Company, Inc. ISBN: 0962781592. Searches: https://www.amazon.com/s?k=0962781592 https://www.google.com/search?q=isbn+0962781592 https://lccn.loc.gov/91203772
Margin (2020/10/22). Ma4: Assumptions (A Wrestling Match). retraice.com. https://www.retraice.com/segments/ma4 Retrieved 24th Oct. 2020.
Retraice (2020/09/07). Re1: Three Kinds of Intelligence. retraice.com. https://www.retraice.com/segments/re1 Retrieved 22nd Sep. 2020.
Retraice (2020/09/11). Re5: Hints From Inside. retraice.com. https://www.retraice.com/segments/re5 Retrieved 22nd Sep. 2020.
Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches: https://www.amazon.com/s?k=978-0134610993 https://www.google.com/search?q=isbn+978-0134610993 https://lccn.loc.gov/2019047498
Simon, H. A. (1996). The Sciences of the Artificial. MIT, 3rd ed. ISBN: 0262691914. Searches: https://www.amazon.com/s?k=0262691914 https://www.google.com/search?q=isbn+0262691914 https://lccn.loc.gov/96012633 Previous editions available at: https://archive.org/search.php?query=The%20sciences%20of%20the%20artificial
1Amdahl (1991)
2Margin (2020/10/22) at 46:45 min.
3Russell & Norvig (2020) p. 10, p. 18.
4Russell & Norvig (2020) p. 18.
5Simon (1996) p. 6.
6Retraice (2020/09/11) p. 1 note 2.
7Retraice (2020/09/07) p. 1 note 4.
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re5: Hints From InsideOn problems of survival: destruction by enemies and fanatics.
Dulles - Craft of Intelligence, Chpt. 15, 'Security in a Free Society', Part 4, final.
Air date: Friday, 11th Sep. 2020, 4pm Pacific/US.1
"George Tenet, director of the US Central Intelligence Agency (CIA) at the beginning of the twenty-first century, summed up the Agency's main mission in three words: 'We steal secrets.' During the Cold War, Allen Dulles, the longest-serving CIA director, wrote that, over the centuries, intelligence organizations had also shown themselves 'an ideal vehicle for conspiracy'."2
We talked yesterday about problems of trust: 'contrived leaks' betrayals, and the role of motivation. Main pointsThis segment is about problems of survival: preventing a group's destruction by enemies (those who would exploit information and trust problems) and by its own fanatics (those who would do anything to solve information and trust problems, even destroy the group's definition). It's based on Allen Dulles's The Craft of Intelligence3 chpt. 15, 'Security in a Free Society'. This is the 4th and final part of our chpt. 15 series.
Retraice works on the question "What's going on out there?" Our point of departure is the concept of intelligence, which seems to have at least three kinds: natural, artificial, and strategic. We're currently working on strategic intelligence, i.e. espionage and its related activities.
We're reading Dulles's chpt. 15 because it might contain hints about the 5% of expensive information4 we want: information about survival—of life, leisure and legislation—against enemies and fanatics, by way of the 5%, the price of which seems to be legwork, empiricism, and competition.
Survival of life, leisure and legislationWe talked yesterday about problems of trust (contrived leaks, betrayals), and before that problems of information (careless leaks, giveaways), as labeled by Dulles.
If giveaways and careless leaks are your team's information problems, and contrived leaks and betrayals are your team's trust problems, what can we say about solutions?
Solutions to information and trust problemsWe can say that the problems need to be mitigated at least. But we can also say that totally solving them is logically impossible: at every moment, an individual mind might be changing; if it changes in certain ways, the team will have new information problems or trust problems. Minds, by definition, are continuously changing.
Here we have a hint of totalitarian logic: If only we (a team's leaders, owners, rulers) could stop minds from changing—or changing in certain ('wrong') ways. The only way to stop minds from changing at all is to destroy them. This is why it's logically impossible to totally solve i- and t- problems: destroyed minds cannot do anything, including problematic things like leaks, giveaways and betrayals.
Hence killing and kinetic warfare are sometimes done even by 'good guys' who have run out of ideas, but are determined to survive.7
But the ways to stop minds from changing in certain ways are many (think: propaganda, rhetoric, fear, inspiration, norms, laws, hiding certain bits of information while endlessly broadcasting others).
Dulles chpt. 15 recapWhat did Dulles want us to know, and what hints can we derive from his words about 'the secret world'8?
Dulles recap:
The importance of espionage: the first or second oldest profession.10
RecapWe're all players on various kinds of teams (some more important than others). As individuals, we want both ourselves and our teams to survive—not just to keep breathing, but to keep and improve the kind of life we've inherited, or built, on top of that breathing, one with some free time, and with freedom from tyranny. From the writings of spymaster Dulles (or a team writing in his name), we've gleaned information about how strategic intelligence contributes to a team's survival, by confronting problems of information and trust, among other things.
NextThis concludes our first series on strategic intelligence. Next: artificial intelligence.
ReferencesAndrew, C. (2018). The Secret World: A History of Intelligence. Yale University Press. ISBN in paperback edition printed as "978-0-300-23844-0 (hardcover : alk. paper)". Searches: https://www.amazon.com/s?k=978-0300238440 https://www.google.com/search?q=isbn+978-0300238440 https://lccn.loc.gov/2018947154
Andrew, C., & Dilks, D. (Eds.) (1984). The Missing Dimension: Governments and Intelligence Communities in the Twentieth Century. Macmillan Publishers LTD (eBook), University of Illinois Press (hbk). ISBN 978-1349072347 (PDF eBook); ISBN 0252011570 (hbk). Citations may be from the PDF or the hbk. eBook available at: https://link.springer.com/content/pdf/10.1007%2F978-1-349-07234-7.pdf Retrieved 20 Aug. 2020.
Ash, T. G. (2020). Hearts don't beat faster for 'the rules-based international order'. Financial Times. 10th Sep. 2020 https://www.ft.com/content/0388ee5d-69aa-41c8-bd3c-8603dfe14a36 Retrieved 11th Sep. 2020.
Dulles, A. (1947). Germany's Underground: The Anti-Nazi Resistance. De Capo. First published 1947; De Capo edition 2000. ISBN: 0306809281. Searches: https://www.amazon.com/s?k=0306809281 https://www.google.com/search?q=isbn+0306809281 https://lccn.loc.gov/99058434
Dulles, A. (2016). The Craft of Intelligence. Lyons Press / Rowman & Littlefield. First published 1963. This edition copyright Joan Buresch Talley, daughter of Dulles. ISBN: 978-1493018796. Searches: https://www.amazon.com/s?k=978-1493018796 https://www.google.com/search?q=isbn+978-1493018796 https://lccn.loc.gov/2016017105 Different editions available at: https://archive.org/search.php?query=The%20Craft%20of%20Intelligence
Harrington, J. (1887). The Commonwealth of Oceana. George Routledge and Sons. First published 1656. This edition 1887. https://archive.org/details/cu31924030366094/page/n9/mode/2up Retrieved 11th Sep. 2020.
Oleson, P. C. (Ed.) (2016). AFIO's Guide to the Study of Intelligence. Association of Former Intelligence Officers, 1st ed. Citations are of the pbk edition, ISBN: 978-0997527308. PDF edition available at: https://www.afio.com/40_guide.htm Retrieved 10th Sep. 2020.
Ransom, H. H. (1984). Secret intelligence in the United States, 1947-1982: the CIA's search for legitimacy. In Andrew & Dilks (1984).
Retraice (2020/09/08). Re2: Tell the People, Tell Foes. retraice.com. https://www.retraice.com/segments/re2 Retrieved 22nd Sep. 2020.
Retraice (2020/xx/xx). Re6: Interface. retraice.com. https://www.retraice.com/segments/re6 Retrieval pending.
Samuelson, R. (2017). A government of laws, not of men. Claremont Review of Books. Fall, 2017. https://claremontreviewofbooks.com/a-government-of-laws-not-of-men/ Retrieved 11th Sep. 2020.
Talbot, D. (2015). The Devil's Chessboard: Allen Dulles, the CIA, and the Rise of America's Secret Government. Harper Perennial. ISBN: 978-0062276179. Searches: https://www.amazon.com/s?k=978-0062276179 https://www.google.com/search?q=isbn+978-0062276179 https://lccn.loc.gov/2015487367
NBC News (1965). The science of spying. [video recording]. 4th May. 1965. https://archive.org/details/gov.archives.arc.614513 Retrieved 11th Sep. 2020. See also: https://www.cia.gov/library/readingroom/docs/CIA-RDP67-00318R000100050001-2.pdf (a script) and https://www.cia.gov/library/readingroom/docs/CIA-RDP82R00025R000500260013-9.pdf (an analysis of the content and public reactions to it) Retrieved 11th Sep. 2020.
Vallee, J. (1979). Messengers of Deception: UFO Contacts and Cults. And/Or Press. ISBN: 0915904381. A different edition available at: https://archive.org/details/MessengersOfDeceptionUFOContactsAndCultsJacquesValle1979/mode/2up
1Retraice works on the question 'What's going on out there?' Our point of departure is the concept of intelligence, which seems to have at least three kinds: natural, artificial, and strategic. Here, we're working on strategic intelligence.
2Andrew (2018) p. 2, citing Ransom (1984) p. 205, who gives no clear source for the quote. UPDATE: After the livestream, we were able to find the source: Dulles (1947), p. 70. Also of note, Dulles's actual words were: "An intelligence service is the ideal vehicle for a conspiracy. [emphasis added] Its members can travel about at home and abroad under secret orders, and no questions are asked. Every scrap of paper in the files, its membership, its expenditure of funds, its contacts, even enemy contacts, are state secrets. Even the Gestapo could not pry into the activities of the Abwehr until Himmler absorbed it. He only succeeded in doing so late in 1943." This update will be mentioned in Segment 6, Retraice (2020/xx/xx).
3Dulles (2016)
4The 5% idea is based on Vallee (1979) p. 68, 'Major Murphy's' model for how to think about strategic intelligence: the most important information in an intelligence contest is likely to be the most expensive, not the most abundant.
5Harrington (1887) p. 16. Harrington, who attributes the idea to Aristotle and Livy, was later cited by John Adams, according to Samuelson (2017).
6Ash (2020) on rules and laws: "An underlying value or principle takes precedence over arbitrary rules, which are often the product of messy bureaucratic, diplomatic and political compromise. Such rules should not be confused with international law, which a UK minister this week outrageously, foolishly and unacceptably said his government will 'break in a specific and limited way'. International law has a very high value and must be upheld. But if what we mean by 'rules' is actually law, then we should say law — a stronger, simpler and more precise word."
7Cf. Retraice (2020/09/08) p. 4, on 'good guys' and their (arguable) "[n]eed for 'the bloodthirsty' " (bad guys).
8Andrew (2018)
9See also NBC News (1965) at 48:04 where Dulles says "I can assure you that the CIA—when I was there as director and I'm quite sure it's the same with Mr. McCone—has given these committees full information about what it's doing, how it's spending its money, and how it operates. When I appeared before them, again and again I've been stopped by members of the Congress, who said, 'We don't want to hear about that, we might talk in our sleep! Don't tell us this!' " Cf. Talbot (2015) p. 551.
10Oleson (2016) p. ii
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Re4: Trust No OneOn problems of trust: 'contrived leaks' and betrayals.
Dulles - Craft of Intelligence, Chpt. 15, 'Security in a Free Society', Part 3.
Air date: Thursday, 10th Sep. 2020, 10 AM Pacific/US.1
He was primarily a messenger and clerk responsible for the distribution and circulation of documents within NSA. What was in these documents may not even have been entirely intelligible to him. But it didn't have to be. All Dunlap had to do was photograph them and make sure that the film reached the Soviet officer handling him. If Dunlap had lived, it is unlikely that he could have recalled more than a small part of the material he passed the Soviets.2
We talked yesterday about three kinds of problems in Dulles's3 Chpt. 15, 'Security in a Free Society': problems of information, problems of trust, and problems of survival. We focused on information problems, 'giveaways' and 'careless leaks'. Main points—unintended problems and problems of trustIf giveaways and careless leaks are your team's unintended information problems, contrived leaks (e.g. whistle-blowers, or press manipulation by group-internal competitors) and betrayals are intended information problems, which are, for your team, better described as trust problems: first picking whom you trust carefully, and second, hardening your team's valuables4, bulwarking, against betrayals—by temporary dissenters, and by permanent enemies.
Problems of trust—'contrived leaks' and betrayalsDulles's two problems of trust are distinguished by the motivations of the actors involved.
'Contrived leaks'—when the motivation is sub-group hostilityHere, the problem of trust is that even a group's friends or allies might see fit to strategically leak privileged information, perhaps for the good of the group directly, or perhaps because the friend or ally thinks their own standing should be higher for the good of the group, and so leaking serves both their own and the group's interests.
This is as much deterrence as hardening: the fewer the people, the harder a group is to compromise; but even if someone in the group has been inserted or turned, they now need to have an escape plan if they're going to leak anything.8
In terms of cases, he gives many. And more generally, he says: "Dunlap's case was one frequently encountered in intelligence history where an insignificant employee, of meager understanding and less education, performing menial tasks but located at a vulnerable point in the internal workings of a highly secret undertaking, can do as much damage as a top-ranking official."11
Lowenthal says this might be what happened with James Angleton after Kim Philby was exposed: "[T]he basic tendency within any intelligence organization (or any organization, for that matter) is to trust its own people, who have been vetted and cleared. They work with one another every day. Familiarity can lead to lowering one's guard or being unwilling to believe that one's own people may be disloyal…. But the alternative behavior—unwarranted suspicion—can be just as debilitating as having a spy in one's midst. James Angleton, who was in charge of the CIA's counterintelligence from 1954 to 1974, became convinced that a Soviet mole—a deeply hidden spy—had penetrated the CIA. Some believe that Angleton was reacting to the fact that his close British associate, Kim Philby, had turned out to be a Soviet agent. Angleton was unable to find the mole, and some believe that he tied the CIA in knots by placing virtually anyone under suspicion. Some suggested that Angleton himself was the mole….14
Compare Peter Grose on the idea of a 'mole': A relatively recent term, and mostly fictional: "The CIA—along with the rest of the world—never succeeded in penetrating the mind of Stalin, and did only marginally better with the Soviet leaders who succeeded him. Acquiring such intelligence would ideally require a 'mole,' a well-connected agent within the Kremlin who could accurately and confidently report out his findings. (The term 'mole' is a relatively recent addition to the lexicon of espionage. It was coined by the British novelist John le Carré.) Though such a boon sometimes appears in the history of intelligence, it generally belongs in the category of what Allen [Dulles] called 'story-book stuff.' "15
Information problems (careless leaks, giveaways) are different from trust problems (contrived leaks, betrayals), based on the intentions, and group memberships, of those involved. Everyone has selfish motives, and most have selfless motives (for the good of the group) as well. Money, ideology, compromise or coercion, and ego, can change motives. But even solving a particular trust problem case can lead to paranoia and profound dysfunction. Strategic intelligence is a world of inherent, complex problems, and any group trying to survive would seem to need subtle, if not complex, solutions to its strategic intelligence problems.
NextWe'll talk about survival, and the role of strategic intelligence generally, and spies in particular.
ReferencesCharney, D. L., & Irvin, J. A. (2016). A guide to the psychology of espionage. In Oleson (2016).
Dulles, A. (2016). The Craft of Intelligence. Lyons Press / Rowman & Littlefield. First published 1963. This edition copyright Joan Buresch Talley, daughter of Dulles. ISBN: 978-1493018796. Searches: https://www.amazon.com/s?k=978-1493018796 https://www.google.com/search?q=isbn+978-1493018796 https://lccn.loc.gov/2016017105 Different editions available at: https://archive.org/search.php?query=The%20Craft%20of%20Intelligence
Grose, P. (1994). Gentleman Spy: The Life of Allen Dulles. Houghton Mifflin. ISBN: 0395516072. Also available at: https://archive.org/details/gentlemanspylife00gros
Hersh, S. M. (2018). Reporter: A Memoir. Vintage / Penguin Random House. ISBN: 978-0307276612. Searches: https://www.amazon.com/s?k=978-0307276612 https://www.google.com/search?q=isbn+978-0307276612 https://lccn.loc.gov/2017051856
Lowenthal, M. M. (2020). Intelligence: From Secrets to Policy. CQ Press / SAGE Publications, 8th ed. ISBN: 978-1544358345. Searches: https://www.amazon.com/s?k=978-1544358345 https://www.google.com/search?q=isbn+978-1544358345 https://lccn.loc.gov/2019027254 Other editions available at: https://archive.org/search.php?query=Intelligence%3A%20From%20Secrets%20to%20Policy
Oleson, P. C. (Ed.) (2016). AFIO's Guide to the Study of Intelligence. Association of Former Intelligence Officers, 1st ed. Citations are of the pbk edition, ISBN: 978-0997527308. PDF edition available at: https://www.afio.com/40_guide.htm Retrieved 10th Sep. 2020.
Retraice (2020/09/11). Re5: Hints From Inside. retraice.com. https://www.retraice.com/segments/re5 Retrieved 22nd Sep. 2020.
Schneier, B. (2003). Beyond Fear: Thinking Sensibly About Security in an Uncertain World. Copernicus Books. ISBN: 0387026207. Searches: https://www.amazon.com/s?k=0387026207 https://www.google.com/search?q=isbn+0387026207 https://lccn.loc.gov/2003051488 Similar edition available at: https://archive.org/details/beyondfearthinki00schn_0
1Retraice works on the question 'What's going on out there?' Our point of departure is the concept of intelligence, which seems to have at least three kinds: natural, artificial, and strategic. Here, we're working on strategic intelligence.
2Dulles (2016) p. 249
3Dulles (2016)
4On 'valuables', cf. Hersh (2018) p. 208 on the CIA's 'family jewels'.
5Dulles (2016) p. 241par21.
6Dulles (2016) p. 238par12.
7Dulles (2016) p. 243.
8Cf. Schneier (2003) pp. 176-178 on the nature and efficacy of deterrence, including its inextricable link to education.
9See Retraice (2020/09/11) p. 3, 'Dulles Recap' point 4.
10Dulles (2016) p. 249par43.
11Dulles (2016) p. p. 249par45.
12Charney & Irvin (2016) p. 465.
13Dulles (2016) p. 250par48.
14Lowenthal (2020) p. 213.
15Grose (1994) p. 469.
For proper formatting (bold, italics, etc.) and graphics (where applicable) see the PDF version. Copyright: 2020 Retraice, Inc.
Ma5: Nobody CaresWhy no one cares about your business, and no one should, and that's a relief.
Air date: Friday, 23rd Oct. 2020, 08 : 40 AM Pacific/US.
1 You You careYou care, in your own way, about your business. But it's good news that no one else does, because there are lots of downsides, and few upsides.
On early adopters, see Ries.1
On wireheading, see Russell.2
There's not enough care to go aroundDo you care about the things you pay for? Probably not, because there's not enough care in a person's life to go around.
On social vs. market norms, see Ariely.3
People who care pay attention. At first that seems great. But what if it goes away? Or what if it becomes a distraction? What if they see things you don't want them to see?
Going live, humilityIf you imagine a big launch, a big day, you're going to do strange things in anticipation of that, like getting strangers' email addresses.
The Dai-ichi Life Insurance saleswoman, Mrs. ShibataAs a counterpoint, see Broughton4 on Mrs. Shibata and her excellence at sales—and getting contact information.
Entrepreneur meet-upsDon't say 'that's a good idea' or 'I'd pay for that', because it's hard to know whether such things are true.
Experiments on customersOn experiments on customers, see Ries.5
On Intuit and Turbotax experiments, see Ries.6
Site note: On Intuit and OKRs, see Doerr.7
2 Friends and Family—'you need to get a job'On telling friends and family about what it's like, see the Cathy Hughes interview:
"That's when I was in a sleeping bag and cooking on a hotplate and washing up in the public bathroom in the radio station…. And when you're starting off in business, you have to be so very careful about who you discuss your hard times with because those who love you and care for you the most like your mother, like your best friend—they'll give you the worst damn advice in the world. 'You need to get a job! You shouldn't be sleeping on the floor in a sleeping bag!' "8
Friends and family pay attention to your business not because they care about it, but because they care about you.
If there were people who caredThe attention might be more distracting than profitable. Do you want more founders, owners, shareholders? Or people who act like they are those things?
You want good shareholders, like Bill Campbell9, not bad shareholders.
And you want your mistakes to be missed and forgotten, not brought to your attention two hours (or two weeks) after you already fixed them.
Fans vs. customersApple crazies are fans, and only customers incidentally. You don't want your fans to convince you something is good when it's not.
FishhooksYou have to get rid of the 50,000 things that will stick into a customer and prevent them from getting to your shop.
Anecdote: Bonefish Grill vs. Chili's.
Customers don't have enough care to overcome the fishhooks themselves.
3 CustomersSite note: we need better words than 'employee', and 'team'.
Customers don't care, mountains don't careCustomers don't care.10
The mountains don't care—but we do.11
You can deal with most dangerous things that 'don't care' by a modest effort. Customers (who don't care) are much harder to deal with. That's (perhaps) the whole subject of business.
4 NobodiesI.e. people who are not in your business, your family, your circle of friends, or your customers or prospective customers.
You'll probably hear a lot from these people, if you're doing something worth doing.
ToxicsConsider the difference between (really) good employees12 and bad employees, toxic people. Toxics are nobodies.
5 Weekend thoughtIt's Friday, so we want to give you something to chew on over the weekend.
Mark Cook's four prudent, unanswerable questionsConsider Ries13 on (Kodak Gallery VP of Products) Mark Cook's four questions:
Compare Jobs (as quoted by Isaacson):
"Some people say, 'Give the customers what they want.' But that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, 'If I'd asked customers what they wanted, they would have told me, 'A faster horse!' ' People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page."14
If you're trying to build something really new, perhaps Cook's questions don't matter.
They don't knowConsider cars vs. horse and buggy: how would the buggy owners have known?
Consider motorcycles vs bicycles: how would the bicyclists have known?
An invention on the spotPowered monitor speakers with LED levels on the front. Right??
6 FridayIt's Friday—time to catch up, get ahead. Go to work.
ReferencesAriely, D. (2009). Predictably Irrational, Revised and Expanded Edition: The Hidden Forces That Shape Our Decisions. Harper Perennial, Kindle ed. ISBN: 978-0061958724. Searches: https://www.amazon.com/s?k=978-0061958724 https://www.google.com/search?q=isbn+978-0061958724 https://lccn.loc.gov/2008273863
Broughton, P. D. (2012). The Art of the Sale: Learning from the Masters About the Business of Life. Penguin. ISBN: 978-1594203329. Searches: https://www.amazon.com/s?k=978-1594203329 https://www.google.com/search?q=isbn+978-1594203329 https://lccn.loc.gov/2011040209
Doerr, J. (2017). Measure What Matters: OKRs: The Simple Idea that Drives 10x Growth. Penguin Random House. ISBN: 978-0241348482. Searches: https://www.amazon.com/s?k=978-0241348482 https://www.google.com/search?q=isbn+978-0241348482 https://lccn.loc.gov/2018002727
Hughes, C., & Raz, G. (2017). Radio One: Cathy Hughes. NPR's How I Built This with Guy Raz. 14th Aug. 2017. https://www.npr.org/2017/09/29/542650845/radio-one-cathy-hughes Retrieved 23rd Oct. 2020.
Isaacson, W. (2011). Steve Jobs. Simon & Schuster. ISBN: 978-1451648539. Searches: https://www.amazon.com/s?k=978-1451648539 https://www.google.com/search?q=isbn+978-1451648539 https://lccn.loc.gov/2011045006
Molenaar, D. (2009). Mountain's Don't Care, But We Do. Mountaineers Books. ISBN: 978-0615293240. Searches: https://www.amazon.com/s?k=978-0615293240 https://www.google.com/search?q=isbn+978-0615293240
Norris, D. (2014). The 7 Day Startup: You Don't Learn Until You Launch. Dan Norris. ISBN: 978-1502472397. Searches: https://www.amazon.com/s?k=978-1502472397 https://www.google.com/search?q=isbn+978-1502472397
Ries, E. (2011). The Lean Startup. Currency / Crown / Penguin Random House. ISBN: 978-0307887894. Searches: https://www.amazon.com/s?k=978-0307887894 https://www.google.com/search?q=isbn+978-0307887894 https://lccn.loc.gov/2011012100
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. ISBN: 978-0525558613. Searches: https://www.amazon.com/s?k=978-0525558613 https://www.google.com/search?q=isbn+978-0525558613 https://lccn.loc.gov/2019029688
Salter, A. (2003). Predators. Basic Books. ISBN: 978-0465071732. Searches: https://www.amazon.com/s?k=978-0465071739 https://www.google.com/search?q=isbn+978-0465071739 https://lccn.loc.gov/2002015846
Sutton, G. (2012). Run Your Own Corporation: How to Legally Operate and Properly Maintain Your Company Into the Future. BZK Press. ISBN: 978-1937832100. Searches: https://www.amazon.com/s?k=978-1937832100 https://www.google.com/search?q=isbn+978-1937832100
Index Bill Campbell, 1 Bonefish Grill, 2Chili's, 2
Dai-ichi Life Insurance, 1
Henry Ford, 2
Intuit, 1
Mark Cook, Kodak Gallery VP of Products, 2
OKRs, 1
Shibata, Mrs., 1 Steve Jobs, 2
Turbotax, 1
wireheading, 1
1Ries (2011) pp. 94-95.
2Russell (2019) pp. 205-208.
3Ariely (2009) p. 75.
4Broughton (2012) p. 81
5Ries (2011) p. 4.
6Ries (2011) p. 33.
7Doerr (2017) p. 102 ff.
8Hughes & Raz (2017) at 24:00 min to 25:20 min.
9Doerr (2017) p. xii and p. 247 ff.
10Norris (2014) p. 39, p. 101, and see also the Audible edition for elaboration and commentary on the text.
11See Salter (2003) pp. 159-160, and Molenaar (2009).
12On 'regular' employees and 'key' employees, and employment agreements and considerations, see Sutton (2012) pp. 84-85.
13Ries (2011) p. 64.
14Isaacson (2011) p. 567
From the publisher's feed