Notes by Retraice

Re15: Trust and Sources NOTES


Listen Later

(The below text version of the notes is for search purposes and convenience.  See the PDF version for proper formatting such as bold, italics, etc., and graphics where applicable. Copyright: 2022 Retraice, Inc.)


Re15: Trust and Sources

Retraice^1

On the neglected basis of all knowledge and belief.

Air date: Wednesday, 25th Nov. 2020, 1:30 PM Pacific/US.

Detection and reports

Perhaps the only difference between something you perceive directly and something somebody tells you is how much you trust it.

Instead of talking about `sources', we can talk about detection and reports. C.f. Russell on knowledge by acquaintance (detection) and knowledge by description (reports).^2

The mirrors of Rjukan, Norway

They don't get enough sunlight at that latitude and near those mountains, so they reflect sunlight into the town using mirrors.^3

If you believe this, it is because reports and trust have convinced you (unless you've been to Rjukan).

Fakes, tech and incentives

While new technologies (e.g. deep-faking) do make fake reports (images, audio, videos) more possible, only incentives can make fakes probable. But as technology gets better, the requisite level of faking-incentive goes down.

Do you trust everything you perceive?

Hallucinations can be profound or barely noticeable.^4 Michael Shermer thought he was abducted by aliens but later concluded he was just riding too much bike.^5 What changed was the trust--he went from trusting earlier detection to trusting later detection and reports from others.

Trust is a mess

There's no clear, widely-known way to think about trust and sources. Adding `detection and reports' doesn't help.

Recall Cynthia Grabo^6 on deception.

A hundred years ago, Walter Lippmann wrestled with this problem.^7 It was essentially the same problem back then that it is today. Edward Bernays also worked on it.^8 Noam Chomsky had lots to say about both of them.^9 See also Ken Thompson from the programming point of view.^10

Even if you have a good system for deciding trust problems^11 , an adversary who knows you're using that system will probably find weaknesses in it and cause you to mistrust. This gets dark.^12 And it's also what computer hacking is about.^13

The concept of trust is scattered throughout the security-related and other literatures, as:
o essential to military hardware^14 o catalyst of intelligence work^15 o prerequisite to espionage^16 o leading to a `deception paradox'^17 o the central problem of counterespionage^18 o most dangerous in the form of `insiders'^19 o the central problem of Internet security^20 o obviating the need for large-scale surveillance and enabling large-scale long-term cooperation^21 o gained by contracts, credit scores and references^22 o gained by deception^23 o gained by self-deception^24 o essential to rationality^25 o essential to science and its division of labor^26 o burden on the trustee^27 o a deciding factor in whether to use a neural network^28 o a dividing line between AI developers and AI users^29 o essential in sales^30 o essential in writing^31

etc. But where is the clear, widely-known way to think about it?

References

Adee, S. (2008). The hunt for the kill switch. IEEE Spectrum, 45(5), 34-39. May 2008.
Primary source and search:
https://ieeexplore.ieee.org/document/4505310 Retrieved 27th Feb. 2022.
https://www.google.com/search?q=%22hunt+for+the+kill+switch%22+pdf

Bernays, E. (1923). Crystallizing Public Opinion. Boni and Liveright (Reprint Ig Publishing). ISBN: 978-1935439264. Searches:
https://www.amazon.com/s?k=978-1935439264
https://www.google.com/search?q=isbn+978-1935439264
https://lccn.loc.gov/2011025346

Bernays, E. (1928). Propaganda. H. Liveright (Reprint by Ig Publishing). ISBN: 978-0970312594. Searches:
https://www.amazon.com/s?k=978-0970312594
https://www.google.com/search?q=isbn+978-0970312594
https://lccn.loc.gov/2004016015

Bernays, E. (1952). Public Relations. University of Oklahoma Press. ISBN: 978-0806114576. Searches:
https://www.amazon.com/s?k=978-0806114576
https://www.google.com/search?q=isbn+978-0806114576
https://lccn.loc.gov/52001707

Broughton, P. D. (2012). The Art of the Sale: Learning from the Masters About the Business of Life. Penguin. ISBN: 978-1594203329. Searches:
https://www.amazon.com/s?k=978-1594203329
https://www.google.com/search?q=isbn+978-1594203329
https://lccn.loc.gov/2011040209

Chomsky, N. (1989). Necessary Illusions: Thought Control in Democratic Societies. South End Press. ISBN: 0896083667. Searches:
https://www.amazon.com/s?k=0896083667
https://www.google.com/search?q=isbn+0896083667
https://lccn.loc.gov/89006074

Chomsky, N. (2017). Requiem for the American Dream: The 10 Principles of Concentration of Wealth & Power. Seven Stories Press. ISBN: 978-1609807368. Searches:
https://www.amazon.com/s?k=978-1609807368
https://www.google.com/search?q=isbn+978-1609807368
https://lccn.loc.gov/2016054121

de Becker, G. (1997). The Gift of Fear: And Other Survival Signals That Protect Us from Violence. Dell / Random House. ISBN: 0440508835. Searches:
https://www.amazon.com/s?k=0440508835
https://www.google.com/search?q=isbn+0440508835
https://lccn.loc.gov/96051051

Dulles, A. (2016). The Craft of Intelligence. Lyons Press / Rowman & Littlefield. First published 1963. This edition copyright Joan Buresch Talley, daughter of Dulles. ISBN: 978-1493018796. Searches:
https://www.amazon.com/s?k=978-1493018796
https://www.google.com/search?q=isbn+978-1493018796
https://lccn.loc.gov/2016017105
Different editions available at:
https://archive.org/search.php?query=The%20Craft%20of%20Intelligence

Erickson, J. (2008). Hacking: The Art of Exploitation. No Starch Press, 2nd ed. ISBN: 978-1593271442. Searches:
https://www.amazon.com/s?k=978-1593271442
https://www.google.com/search?q=isbn+978-1593271442
https://lccn.loc.gov/2007042910

Gopnik, A., Meltzoff, A. N., & Kuhl, P. K. (1999). The Scientist in the Crib: What Early Learning Tells Us About the Mind. Perennial / HarperCollins. ISBN: 0688159885. Searches:
https://www.amazon.com/s?k=0688159885
https://www.google.com/search?q=isbn+0688159885
https://lccn.loc.gov/99024247

Grabo, C. M. (2002). Anticipating Surprise: Analysis for Strategic Warning. Center for Strategic Intelligence Research. ISBN: 0965619567
https://www.ni-u.edu/ni_press/pdf/Anticipating_Surprise_Analysis.pdf Retrieved 7th Sep. 2020.

Hawley, K. (2012). Trust: A Very Short Introduction. Oxford University Press. ISBN: 9780199697342. Searches:
https://www.amazon.com/s?k=9780199697342
https://www.google.com/search?q=isbn+9780199697342
https://lccn.loc.gov/2012406649

Herman, E. S., & Chomsky, N. (1988). Manufacturing Consent: The Political Economy of the Mass Media. Pantheon. ISBN: 0679720340. Searches:
https://www.amazon.com/s?k=0679720340
https://www.google.com/search?q=isbn+0679720340
https://lccn.loc.gov/88042614

Kidder, T., & Todd, R. (2013). Good Prose. Random House Publishing Group, kindle edition ed. ISBN: 9780679604723. Searches:
https://www.amazon.com/s?k=9780679604723
https://www.google.com/search?q=isbn+9780679604723
https://lccn.loc.gov/2012021165

Lippmann, W. (1920). Liberty and the News. Harcourt, Brace and Howe (Leopold Reprint). No ISBN. eBook and searches:
https://books.google.com/books?id=Df-SzcLRcAIC Retrieved 24th Feb. 2022.
https://www.amazon.com/s?k=Liberty+and+the+News+Lippmann
https://www.google.com/search?q=liberty+and+the+news+lippmann
https://lccn.loc.gov/20004814

Oleson, P. C. (Ed.) (2016). AFIO's Guide to the Study of Intelligence. Association of Former Intelligence Officers, 1st ed. Citations are of the pbk. edition, ISBN: 978-0997527308. PDF edition available at:
https://www.afio.com/40_guide.htm Retrieved 10th Sep. 2020.

Peterson, M. (2017). An Introduction to Decision Theory. Cambridge University Press, 2nd ed. ISBN: 9781316606209. Searches:
https://www.amazon.com/s?k=9781316606209
https://www.google.com/search?q=isbn+9781316606209
https://lccn.loc.gov/2016057387

Pinker, S. (2011). The Better Angels of Our Nature: Why Violence Has Declined. Penguin Publishing Group. ISBN: 978-0143122012. Searches:
https://www.amazon.com/s?k=978-0143122012
https://www.google.com/search?q=isbn+978-0143122012
https://lccn.loc.gov/2011015201

Retraice (2020/09/07). Re1: Three Kinds of Intelligence. retraice.com.
https://www.retraice.com/segments/re1 Retrieved 22nd Sep. 2020.

Russell, B. (1912). The Problems of Philosophy. Dover (Reprinted 1999). ISBN: 0486406741. Searches:
https://www.amazon.com/s?k=0486406741
https://www.google.com/search?q=isbn+0486406741
https://lccn.loc.gov/98033214

Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches:
https://www.amazon.com/s?k=978-0134610993
https://www.google.com/search?q=isbn+978-0134610993
https://lccn.loc.gov/2019047498

Sacks, O. (2012). Hallucinations. Vintage Books (Reprinted 2013). ISBN: 978-0307947437. Searches:
https://www.amazon.com/s?k=978-0307947437
https://www.google.com/search?q=isbn+978-0307947437
https://lccn.loc.gov/2012002877

Salter, A. (2003). Predators. Basic Books. ISBN: 978-0465071732. Searches:
https://www.amazon.com/s?k=978-0465071739
https://www.google.com/search?q=isbn+978-0465071739
https://lccn.loc.gov/2002015846

Schneier, B. (2003). Beyond Fear: Thinking Sensibly About Security in an Uncertain World. Copernicus Books. ISBN: 0387026207. Searches:
https://www.amazon.com/s?k=0387026207
https://www.google.com/search?q=isbn+0387026207
https://lccn.loc.gov/2003051488
Similar edition available at:
https://archive.org/details/beyondfearthinki00schn_0

Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. W. W. Norton. ISBN: 9780393244816. Searches:
https://www.amazon.com/s?k=9780393244816
https://www.google.com/search?q=isbn+9780393244816
https://lccn.loc.gov/2014048365

Shermer, M. (2005). Abducted! Imaginary traumas are as terrifying as the real thing. Scientific American. Feb. 2005.
https://michaelshermer.com/sciam-columns/abducted/ Retrieved 25th Nov. 2020.
https://www.scientificamerican.com/article/abducted/ Retrieved 28th Feb. 2022.

Simler, K., & Hanson, R. (2018). The Elephant in the Brain: Hidden Motives in Everyday Life. Oxford University Press. ISBN: 9780190495992. Searches:
https://www.amazon.com/s?k=9780190495992
https://www.google.com/search?q=isbn+9780190495992
https://lccn.loc.gov/2017004296

Thompson, K. (1984). Reflections on trusting trust. Communications of the ACM, 27(8), 761-763. Aug. 1984.
https://doi.org/10.1145/358198.358210 Also available at:
https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf Retrieved 4th Dec. 2020.

Footnotes

^1 https://www.retraice.com/retraice

^2 Russell (1912) p. 31 ff.

^3 https://www.google.com/search?q=mirrors+Rjukan

^4 Sacks (2012).

^5 Shermer (2005). See also Sacks (2012) p. 42 ff. on Shermer's case.

^6 Grabo (2002) p. 119 ff. See also Retraice (2020/09/07).

^7 See Lippmann (1920) e.g. pp. 37-38, where he talks about second, third and forth hand knowledge, the problem of seeing for oneself, and the unreliability of such methods.

^8 Bernays (1928) pp. 54-55 where trust is implied by his discussions of `authority' and `loyalty'; Bernays (1923) p. 191 on "the most valuable thing" being "faith and trust"; Bernays (1952) p. 112 on PR and suspicion of businessmen.

^9 Among many examples, see Herman & Chomsky (1988) p. xi, p. 332; Chomsky (1989) pp. 16-17; Chomsky (2017) pp. 124-125.

^10 Thompson (1984) on trusting statements, programs and people.

^11 Is this a hint, the word `deciding'? Are all trust problems really decision problems?

^12 Salter (2003) p. 103; Pinker (2011) p. 484.

^13 Erickson (2008) p. 1. The implication is this: If we trust a system that we don't fully understand we become vulnerable to events and actions by others that we cannot anticipate.

^14 Oleson (2016) p. 275 and Adee (2008).

^15 Oleson (2016) p. 231.

^16 Oleson (2016) p. 470.

^17 Oleson (2016) p. 404.

^18 Dulles (2016) pp. 143-144.

^19 Schneier (2003) pp. 62-63.

^20 Schneier (2015) pp. 181-183.

^21 Simler & Hanson (2018) p. 271.

^22 Simler & Hanson (2018) p. 271.

^23 de Becker (1997) p. 59.

^24 Simler & Hanson (2018) pp. 81-83.

^25 Peterson (2017) p. 10 and pp. 266-267.

^26 Gopnik et al. (1999) p. 160.

^27 Hawley (2012) p. 14.

^28 Russell & Norvig (2020) pp. 711-712.

^29 Russell & Norvig (2020) p. 996 ff.

^30 Broughton (2012) p. 291 (index).

^31 Kidder & Todd (2013) location 129.

 

...more
View all episodesView all episodes
Download on the App Store

Notes by RetraiceBy Retraice, Inc.