(The below text version of the notes is for search purposes and convenience. See the PDF version for proper formatting such as bold, italics, etc., and graphics where applicable. Copyright: 2022 Retraice, Inc.)

Re73: Gradients and Partial Derivatives Part 4 (AIMA4e pp. 119-122)

retraice.com

The limits that define our gradient.

Air date: Wednesday, 7th Dec. 2022, 11:00 PM Eastern/US.

We're focusing on the math and code of AIMA4e^1 right now, December 2022. This is in service of our plan to deep-dive the book from Jan.-Jun., 2023. DISCLAIMER: The below mathematics cannot be trusted; it's a student's attempt, not an expert's.

Our gradient equation for the airport problem^2 --three airport locations, each with two coordinates:In mathematics you don't understand things. You just get used to them. --John von Neumann*

() Nablaxf = \partial-f, \partial-f, \partial-f, \partial-f, \partial-f, \partial-f \partialx1 \partialy1 \partialx2 \partialy2 \partialx3 \partialy3

The gradient^3 is like a six-dimensional `slope' in our six-dimensional solution space at a guess `point' (vector) x = (x,y,x,y,x,y). Each scalar -\partialf- \partialxi|yi will be a positive number, negative number, or zero.

To minimize the function value, it seems we'll want to increment each independent variable whichever way (increasing or decreasing) is opposite to the sign of its partial derivative value (its `rate'), i.e. by a positive number if the scalar is negative, a negative number if the scalar is positive, and perhaps not at all if its value is zero.

But we can't calculate the value of f for any input without taking the coordinates of the cities, determining the straight-line distances using geometry, and squaring and summing those distances to arrive at our `score' or `cost' that we want to minimize by changing (improving) the airport locations. We'll work on this later.

The partial derivatives expressed as limits:

----- Thepartialderivative of \partial-f = lim f(x1-+h,y1,x2,y2,x3,y3)--f(x) f with respectto x1: \partialx1 h->0 h ----- Thepartialderivative of \partial-f = lim f(x1,y1+-h,x2,y2,x3,y3)--f(x) f with respectto y1: \partialy1 h->0 h ----- Thepartialderivative of \partial-f f(x1,y1,x2+-h,y2,x3,y3)--f(x) f with respectto x2: \partialx2 = lih->m0 h Thepartialderivative of \partial-f f(x1,y1,x2,y2+h,x3,y3)--f(x) f with respectto y2: \partialy2 = lih->m0 h Thepartialderivative of \partial f f(x1,y1,x2,y2,x3+-h,y3)- f(x) f with respectto x3: \partialx3 = lih->m0 ------------h------------ Thepartialderivative of \partial f f(x1,y1,x2,y2,x3,y3+-h)- f(x) f with respectto y3: \partialy- = lih->m0 ------------h------------ 3

_

*Quoted in Scheinerman (2011), p. iv. See also Zukav (1984) p. 208.

References

Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge University Press. ISBN: 978-1108455145. https://mml-book.github.io/ Searches: https://www.amazon.com/s?k=9781108455145 https://www.google.com/search?q=isbn+9781108455145 https://lccn.loc.gov/2019040762

Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th ed. ISBN: 978-0134610993. Searches: https://www.amazon.com/s?k=978-0134610993 https://www.google.com/search?q=isbn+978-0134610993 https://lccn.loc.gov/2019047498

Scheinerman, E. R. (2011). Mathematical Notation: A Guide for Engineers and Scientists. Independently published. ISBN: 978-1466230521. Searches: https://www.amazon.com/s?k=9781466230521 https://www.google.com/search?q=isbn+9781466230521

Zukav, G. (1984). The Dancing Wu Li Masters: An Overview of the New Physics. Bantam, english language ed. ISBN: 055326382X. https://archive.org/details/dancingwulimaste00zuka_0/page/n7/mode/2up Searches: https://www.amazon.com/s?k=055326382X https://www.google.com/search?q=isbn+055326382X https://lccn.loc.gov/78025827

Footnotes

^1 Russell & Norvig (2020).

^2 Russell & Norvig (2020) p. 120.

^3 The Nabla[x ]f notation comes from Deisenroth et al. (2020) p. 127.