Data Science Decoded

By Mike E

We discuss seminal mathematical papers (sometimes really old 😎 ) that have shaped and established the fields of machine learning and data science as we know them today. The goal of the podcast is to ... more

Download on the App Store

Download on the App Store

Get it on Google Play

FAQs about Data Science Decoded:

How many episodes does Data Science Decoded have?

The podcast currently has 12 episodes available.

Data Science Decoded episodes:

September 28, 2024 Kolmogorov complexity: "3 approaches to the quantitative definition of information", A. Kolmogorov (1965) - Part 1
In the 12th episode we review the first part of Kolmogorov's seminal paper:
"3 approaches to the quantitative definition of information’." Problems of information transmission 1.1 (1965): 1-7.
The paper introduces algorithmic complexity (or Kolmogorov complexity), which measures the amount of information in an object based on the length of the shortest program that can describe it.
This shifts focus from Shannon entropy, which measures uncertainty probabilistically, to understanding the complexity of structured objects.

Kolmogorov argues that systems like texts or biological data, governed by rules and patterns, are better analyzed by their compressibility—how efficiently they can be described—rather than by random probabilistic models.
In modern data science and AI, these ideas are crucial. Machine learning models, like neural networks, aim to compress data into efficient representations to generalize and predict.
Kolmogorov complexity underpins the idea of minimizing model complexity while preserving key information, which is essential for preventing overfitting and improving generalization.
In AI, tasks such as text generation and data compression directly apply Kolmogorov's concept of finding the most compact representation, making his work foundational for building efficient, powerful models.
This is part 1 out of 2 episodes covering this paper
...more
39min
September 20, 2024 "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain", Frank Rosenblatt, 1958
Frank Rosenblatt's 1958 paper, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," introduces the perceptron, an early neural network model inspired by how the brain stores and processes information.
Rosenblatt explores two theories: one where sensory data is stored as coded representations, and another, which he advocates, where learning occurs through forming new neural connections.

The perceptron illustrates this connectionist approach by mimicking how neurons process input and reinforce connections based on experience.
The perceptron operates by passing sensory input through a network of neurons, where weights on connections adjust with each stimulus, enabling the system to recognize patterns and classify information.

Rosenblatt emphasizes the probabilistic nature of learning in the perceptron, which mirrors how biological systems might generalize and adapt based on exposure to different stimuli. His model serves as a theoretical framework for understanding both biological and artificial neural systems.
The paper's significance to modern data science lies in its foundational role in developing machine learning. The perceptron model directly influenced the creation of more advanced neural networks, including today's deep learning models.

Though limited in handling complex, non-linear data, the perceptron established key principles—such as weighted connections and learning from data.
...more
1h 4min
September 12, 2024 The original principal component analysis (PCA) paper, Harold Hotelling (1935)
Hotelling, Harold. "Analysis of a complex of statistical variables into principal components." Journal of educational psychology 24.6 (1933): 417.

This seminal work by Harold Hotelling on PCA remains highly relevant to modern data science because PCA is still widely used for dimensionality reduction, feature extraction, and data visualization.
The foundational concepts of eigenvalue decomposition and maximizing variance in orthogonal directions form the backbone of PCA, which is now automated through numerical methods such as Singular Value Decomposition (SVD).
Modern PCA handles much larger datasets with advanced variants (e.g., Kernel PCA, Sparse PCA), but the core ideas from the paper—identifying and interpreting key components to reduce dimensionality while preserving the most important information—are still crucial in handling high-dimensional data efficiently today.
...more
56min
September 10, 2024 "The unreasonable effectiveness of mathematics in the natural sciences." , Wigner, Eugene P., (1960)
In this special episode, Daniel Aronovich joins forces with the 632 nm podcast.
In this timeless paper Wigner reflects on how mathematical concepts, often developed independently of any concern for the physical world, turn out to be remarkably effective in describing natural phenomena.

This effectiveness is "unreasonable" because there is no clear reason why abstract mathematical constructs should align so well with the laws governing the universe.
Full paper is at our website:
https://datasciencedecodedpodcast.com/episode-9-the-unreasonable-effectiveness-of-mathematics-in-natural-sciences-eugene-wigner-1960
...more
1h 25min
September 04, 2024 The Turing test by Alan Turing, "Computing machinery and intelligence" (1950)
This paper is a foundational text in the field of artificial intelligence (AI) and explores the question: "Can machines think?"
Turing introduces what is now known as the "Turing Test" as a way to operationalize this question, he called it the imitation game.
Are there imaginable digital computers that could perform well in the imitation game?
The imitation game involves an interrogator trying to distinguish between a human and a machine based on their responses to various questions.
Turing argues that if a machine could perform well enough in this game to be indistinguishable from a human, then it could be said to "think." He explores various objections to the idea that machines can think, including theological, mathematical, and arguments from consciousness.

Turing addresses each objection, ultimately suggesting that machines can indeed be said to think if they can perform human-like tasks, especially those that involve reasoning, learning, and language.
...more
55min
August 12, 2024 "The use of multiple measurements in taxonomic problems.", Fisher R.A., (1936)
This paper introduced linear discriminant analysis(LDA), a statistical technique that revolutionized classification in biology and beyond.

Fisher demonstrated how to use multiple measurements to distinguish between different species of iris flowers, laying the foundation for modern multivariate statistics.

His work showed that combining several characteristics could provide more accurate classification than relying on any single trait.

This paper not only solved a practical problem in botany but also opened up new avenues for statistical analysis across various fields.

Fisher's method became a cornerstone of pattern recognition and machine learning, influencing diverse areas from medical diagnostics to AI.

The iris dataset he used, now known as the "Fisher iris" or "Anderson iris" dataset, remains a popular example in data science education and research.

...more
48min
August 07, 2024"On the problem of the most efficient tests of statistical hypotheses", Neyman and Pearson, (1933)
This paper is considered one of the foundational works in modern statistical hypothesis testing.
Key insights and influences:
Neyman-Pearson Lemma: The paper introduced the Neyman-Pearson Lemma, which provides a method for finding the most powerful test for a simple hypothesis against a simple alternative.

Type I and Type II errors: It formalized the concepts of Type I (false positive) and Type II (false negative) errors in hypothesis testing.

Power of a test: The paper introduced the concept of the power of a statistical test, which is the probability of correctly rejecting a false null hypothesis.

Likelihood ratio tests: It laid the groundwork for likelihood ratio tests, which are widely used in modern statistics.

Optimal testing: The paper provided a framework for finding optimal statistical tests, balancing the tradeoff between Type I and Type II errors.
These concepts have had a profound influence on modern statistical theory and practice, forming the basis of much of classical hypothesis testing used today in various fields of science and research.
...more
57min
July 30, 2024 "A Mathematical Theory of Communication", Shannon, C. E., (1948) - Part 3
Shannon, Claude Elwood. "A mathematical theory of communication." The Bell system technical journal 27.3 (1948): 379-423.
Part 3/3.
The paper fundamentally reshapes how we understand communication.
The paper introduces a formal framework for analyzing communication systems, addressing the transmission of information with and without noise. Key concepts include the definition of information entropy, the logarithmic measure of information, and the capacity of communication channels.
In the third part we go over the Fundamental theorem of the noisy and noiseless channel!
Full breakdown of the paper with math and python code is at our website:
https://datasciencedecodedpodcast.com/
...more
1h 8min
July 21, 2024 "A Mathematical Theory of Communication", Shannon, C. E., (1948) - Part 2
Shannon, Claude Elwood. "A mathematical theory of communication." The Bell system technical journal 27.3 (1948): 379-423.
Part 2/3.
The paper fundamentally reshapes how we understand communication.
The paper introduces a formal framework for analyzing communication systems, addressing the transmission of information with and without noise. Key concepts include the definition of information entropy, the logarithmic measure of information, and the capacity of communication channels.
Shannon demonstrates that information can be efficiently encoded and decoded to maximize the transmission rate while minimizing errors introduced by noise. This work is pivotal today as it underpins digital communication technologies, from data compression to error correction in modern telecommunication systems.
Full breakdown of the paper with math and python code is at our website:
https://datasciencedecodedpodcast.com...
This is the second part out of 3, as the paper is quite long!
...more
42min
July 16, 2024 "A Mathematical Theory of Communication", Shannon, C. E., (1948) - Part 1
Shannon, Claude Elwood. "A mathematical theory of communication." The Bell system technical journal 27.3 (1948): 379-423.
Part 1/3.
The paper fundamentally reshapes how we understand communication.
The paper introduces a formal framework for analyzing communication systems, addressing the transmission of information with and without noise. Key concepts include the definition of information entropy, the logarithmic measure of information, and the capacity of communication channels.
Shannon demonstrates that information can be efficiently encoded and decoded to maximize the transmission rate while minimizing errors introduced by noise. This work is pivotal today as it underpins digital communication technologies, from data compression to error correction in modern telecommunication systems.
Full breakdown of the paper with math and python code is at our website:
https://datasciencedecodedpodcast.com...
This is the first part out of 3, as the paper is quite long!
...more
42min

FAQs about Data Science Decoded:

How many episodes does Data Science Decoded have?

The podcast currently has 12 episodes available.