Today I’m speaking with Yury Gorishniy about the state of the competition between Deep Learning and Gradient Boosted Decision Trees when it comes to tabular datasets, and about a recent paper he published that seems to take a stab at improving the state of deep learning on tabular datasets.

We discuss whether or not there exists a gap between deep learning and gradient boosted decision trees, what the future of a gap might look like, and the extent to which the embedding of numerical features can give deep learning architectures a necessary boost in performance.

Two of his recent papers are useful in this discussion:

• On Embeddings for Numerical Features in Tabular Deep Learning - https://arxiv.org/abs/2203.05556
• Revisiting Deep Learning Models for Tabular Data - https://arxiv.org/abs/2106.11959

You can find Yury in the following places:

• GitHub - https://github.com/Yura52
• Twitter - https://twitter.com/YuraFiftyTwo

Enjoy!

Today we’ll be speaking with Julian Morimoto about missing data, its impact on the reliability of statistical inference, and two theorems that he recently discovered using concepts from real analysis about what guarantees we can expect, at the limit of arbitrarily large data sets.

Julian has a background in math, and studied law at Harvard Law School and he speaks about the unique challenges of adopting machine learning in the legal world due to the various mechanisms of missingness in confidential documents.

If you're interested in learning more about missing data and statistical inference, here are several resources to help you get started:

• GOV2001 - Harvard: https://projects.iq.harvard.edu/gov2001/home
• One hour course by Dr. Gary Keng on missing data on YouTube: https://www.youtube.com/watch?v=qlPs8Ioa56Y
• Julian’s paper: https://arxiv.org/abs/2112.09275v4
• Statistical Analysis with Missing Data book: https://www.amazon.com/Statistical-Analysis-Missing-Roderick-Little/dp/0471183865?

In this episode, Adam introduces the show, the motivations for it, and why and how you should participate.

Today we're speaking with Luigi Quaranta about the state of reproducibility in machine learning.

Luigi published a taxonomy of support for reproducibility by various tools in the space and together we’re exploring the need for reproducibility, challenges and limitations, how to evaluate opportunities for improving your current systems, and what the future might hold.

A few papers would be relevant here.

The first is A Taxonomy of Tools for Reproducible Machine
Learning Experiments: http://ceur-ws.org/Vol-3078/paper-81.pdf

and the second is a study of how to make Jupyter notebook code of high quality, Eliciting Best Practices for Collaboration with Computational Notebooks: https://dl.acm.org/doi/abs/10.1145/3512934.

In this episode, Adam speaks with Doctor Torsten Ensslin about simulating reputation networks and their manipulation using Information Theory.

Torsten is an Astrophysicist and cosmologist at the Max Plank institute, where he’s held many titles and positions. His current scientific work investigates theoretical cosmology and information field theory. 

As he discusses in this episode, Torsten co-created a simulation of how reputation propagates through a network of agents and how those agents can manipulate their communication strategies, through lies, targeted deceit, propaganda, and many other tactics, in an effort to achieve domination over their networks.

One of the theses of Torsten’s work is that in order to understand the impact of our AI on our existing systems, it’s important to have a deeper understanding of those systems to begin with. His simulation is a first step in that direction: an attempt to model the interaction of various agents according to information theoretic principles and study the consequent emergent phenomena.

There are two versions of the paper.

The first is a shorter summary of the work, light on the math, and can be found here: https://psyarxiv.com/wqcmb/.

The longer version of the paper can be found here: https://arxiv.org/pdf/2106.05414.pdf.

Enjoy!

In this episode, Adam discusses Multi-objective optimization with Laurent Parmentier.

Laurent works at OVHCloud, most recently as a data scientist but previously in various software engineering roles. He published his thesis on AutoML at OVHCloud, and had previously released a paper titled TPOT-SH: A Faster Optimization Algorithm to Solve the AutoML Problem on Large Datasets.

The conversation centers around two classic papers in AutoML:

• Multi-Objective Automatic Machine Learning with AutoxgboostMC by Pfisterer et al: https://arxiv.org/abs/1908.10796
  
  
• An ADMM Based Framework for AutoML Pipeline Configuration by Sijia Liu, Parikshit Yam et al: https://arxiv.org/abs/1905.00424

Enjoy!

This episode launches us into the deep waters of ML interpretability with Jessica Schrouff.

Jessica is a Senior Research Scientist at Google Research working on machine learning for healthcare. Before joining Google in 2019, she was a postdoctoral fellow at University College London (UK) and Stanford University (USA), developing machine learning techniques for neuroscience discovery and clinical predictions.

Throughout her career, her interests have lied not only in the technical advancement of machine learning methods, but also in critical aspects of their deployment such as their credibility, fairness, robustness or interpretability.

As you’ll hear, we are orbiting Jessica’s latest paper titled “Best of both worlds: local and global explanations with human-understandable concepts”. You will find lots more reference material in the show notes on the website.

Jessica, thank you for joining us!

This is a conversation between data scientist Ankush Garg, from Telepath, and fourth-year Ph.D. student Iman Mirzadeh and they’ll be talking about Continual Learning and about Iman’s paper titled “Architecture Matters in Continual Learning”.

Iman is interested in Artificial General Intelligence and so he’s researching systems that can over time develop increasingly more complex skills and a richer body of knowledge.

Iman received his Bachelor’s degree in Computer Engineering from the University of Tehran and his Master’s degree from Washington State University and that’s where he’s currently a research assistant.

Today we’re talking about MLOps - with our guide Georgios Symeonidis and we’ll be orienting around a recent paper he published titled “MLOps - Definitions, Tools and Challenges”.

Georgios studied electrical and computer engineering at Democritus University of Thrace at Xanthi, in Greece. He specialized in information and electronics.

He’s also worked as a research engineer at Athena Research and Innovation Center - studying ML systems in production. For his PhD, he’s researching Reliable Machine Learning pipelines for mature MLOps systems at International Hellenic University in Greece.

This episode covers the relationship between Curriculum Learning and AutoML with Lucas Nildaimon dos Santos Silva.

Lucas is a data scientist at americanas s.a., in Brazil, and is currently pursuing a Ph.D. in computer science from the Federal University of São Carlos where he also did his master’s. He’s researching machine learning and NLP and has recently published a paper on Curriculum Learning in AutoML systems, found below.

https://ieeexplore.ieee.org/document/9680164