The Information

Publisher's link for The Theory That Would Not Die by Sharon Bertsch McGrayne

Nick introduces a new episode type that may end up dominating the feed, at least for the near future.

We wrap the season with a recap of what we've covered and my thoughts about how these ideas are related to a very general way of analyzing data.

We discuss connections between betting and information theory and how this framework for thinking about how being able to accurately estimate probabilities might apply more generally to decisions that we make.

We discuss basic concepts of information theory as well as some examples of communication processes and compression techniques. The sources for this episode are the following books:

https://www.amazon.com/Information-Theory-Tutorial-Introduction-2nd-dp-1739672704/dp/1739672704/ref=dp_ob_title_bk

https://www.amazon.com/Introduction-Information-Theory-Symbols-Mathematics/dp/0486240614/ref=tmm_pap_swatch_0?_encoding=UTF8&qid=1687105608&sr=1-1

I discuss the work of Judea Pearl and how it fits in with the other topics we've discussed this season. We get to vilify, yet again, our old friend Ronald Fisher.

We compare machine learning to traditional programming and talk about some of the core challenges associated with doing machine learning.

We talk more about data generating processes and how commonalities between the mathematical structure of diverse physical phenomena suggest the presence of a "second dimension" in how we might think about organizing physical phenomena. Can probabilistic programming come to the rescue and provide a quite general framework for analyzing data? Let's see what you think after listening to today's episode.

A discussion of the meaning of probability and how it fits in with our other topics this season.