Machine Learning Cafe

In this episode, I talked with Curtis Northcutt about his application cleanlab, with which you can find label errors in your dataset. Cleanlab computes cross-validated probabilities, the confident joint, and the statistics used in uncertainty estimation for dataset labels, and it ranks and sorts the labels by the probabilities of error, so you can easily find them in your dataset.

Curtis' website: https://www.curtisnorthcutt.com/

Curtis on LinkedIn: https://www.linkedin.com/in/cgnorthcutt/

Cleanlab on GitHub: https://github.com/cgnorthcutt/cleanlab

Cleanlab's blog: https://l7.curtisnorthcutt.com/cleanlab-python-package

White Papers: https://arxiv.org/abs/1911.00068 https://arxiv.org/abs/1705.01936

Music by Curtis (PomDP the PhD rapper): https://soundcloud.com/thephdrapper/bars-on-bars https://soundcloud.com/thephdrapper/crown https://soundcloud.com/thephdrapper/dub-dub https://open.spotify.com/album/2Fjg3zF8PGEg9WWNoeyx3X

---General Info---

Podcast's Website: http://machinelearningcafe.org/ Host's LinkedIn: https://www.linkedin.com/in/miklostoth/ Email of the host: [email protected]

---Copyright Info--- Music is from https://filmmusic.io, intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro: "Bars on Bars" by Curtis Northcutt (with his explicit allowance to play)

In this episode, we interview Less Wright, who has set 13 records on the fast.ai leaderboard, and we talk about one of his tricks is using cutting-edge optimizers, he also developed one, which is called Ranger. He talks also about different strategies of using Deep Learning optimizers, it is worth to take those into consideration. We also tested Ranger and in our case, it also outperformed Adam/RAdam variants. Co-host is Levente Szabados.

Less' LinkedIn: https://www.linkedin.com/in/less-wright-22b59017/

Ranger's article: https://medium.com/@lessw/how-we-beat-the-fastai-leaderboard-score-by-19-77-a-cbb2338fab5c

Less' medium articles: https://medium.com/@lessw His article about Mixnet he mentions: https://medium.com/@lessw/meet-mixnet-google-brains-new-state-of-the-art-mobile-ai-architecture-bd6c37abfa3a

His Github: https://github.com/lessw2020

Co-Host: Levente Szabados https://www.linkedin.com/in/levente-szabados-76334728/

---General Info---

Podcast's Website: http://machinelearningcafe.org/ Host's LinkedIn: https://www.linkedin.com/in/miklostoth/ Email of the host: [email protected]

---Copyright Info--- Music is from https://filmmusic.io, intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)

In this episode, we are focusing on Deep Learning Optimizers, the different Gradient Descent Variants from vanilla GD to RAdam and Ranger. Levente tells us the story of GD from the simplest ones to the newest ones. This is part 1.

Levente's Linkedin URL: https://www.linkedin.com/in/levente-szabados-76334728/

An overview of gradient descent optimization algorithms: https://arxiv.org/pdf/1609.04747.pdf

Lookahead optimizer algorithm: https://arxiv.org/pdf/1907.08610.pdf

Ranger optimizer by Less Wright: https://medium.com/@lessw/new-deep-learning-optimizer-ranger-synergistic-combination-of-radam-lookahead-for-the-best-of-2dc83f79a48d

---Copyright Info--- Music is from https://filmmusic.io, intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)

In this episode, we interview Renee Ahel, who is a Lead AI Expert at Cirtuo and a freelance data scientist in Croatia. He is dealing with Machine Learning since 2003, and recently he is working on Tree-based methods, with which he solves procurement problems. He is also dealing with Expert systems, you will hear why. His motto is: You should choose the most efficient tool for the problem you have, regardless of whether it is in fashion or not.

AI applications in procurement: https://sievo.com/resources/ai-in-procurement

Expert systems https://en.wikipedia.org/wiki/Expert_system

Kraljic matrix https://www.forbes.com/sites/jwebb/2017/02/28/what-is-the-kraljic-matrix/#2d004b38675f

Bayes network powered PCOS (polycystic ovary syndrome) diagnostics - implemented by Croatian company Vingd for the Berlin startup Clue: https://techcrunch.com/2019/09/04/period-app-clue-hopes-to-predict-if-you-have-pcos/

Renee Ahel on LinkedIn: https://www.linkedin.com/in/reneeahel/

Data Science Croatia: https://www.meetup.com/DataScienceCroatia/

In this Episode we talked about the deep neural networks and the spectral density of each layer's weights. It turns out, you can predict the accuracy ( and many more) with the WeigthWatcher application. We talk about the 5+1 Phases of learning and Heavy Tailed Self Regularization.

Charles Martin, PhD on LinkedIn: https://www.linkedin.com/in/charlesmartin14/

During the episode we talked about these VC Theory: https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_theory

Why Deep Learning works? post by Charles Martin, 2015 https://calculatedcontent.com/2015/03/25/why-does-deep-learning-work/ Presentation at Berkeley: Why Deep Learning works? by Charles Martin, 2016 https://www.youtube.com/watch?v=fHZZgfVgC8U

Several Papers written by Charles Martin and Michael Mahoney: https://arxiv.org/search/?query=%22Charles+H.+Martin%22&searchtype=author&abstracts=show&order=-announced_date_first&size=50

Newest blog post about weightwatcher, 2019 December: https://calculatedcontent.com/2019/12/03/towards-a-new-theory-of-learning-statistical-mechanics-of-deep-neural-networks/

WeightWatcher on GitHub: https://github.com/CalculatedContent/WeightWatcher easy installation for python users: pip install weightwatcher

How to reach out: https://calculationconsulting.com/ [email protected]

To access their slack channel please contact Charles first.

---Copyright Info--- Music is from https://filmmusic.io , intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)

In this episode, we talked about the idea behind GANs in general and two special types of GANs: Progressively Growing GANs and Style GANs.

LinkedIn URL of Alexandr Honchar: https://www.linkedin.com/in/alexandr-honchar-4423b962/

Companies where Alex works: http://neurons-lab.com/ http://mawi.band/

Facebook: https://www.facebook.com/rachnogstyle

Blog about ECG pipeline: https://medium.com/mawi-band/towards-ai-based-only-biosignal-analysis-pipeline-39e6e31244a6

Progressively Growing GANs: https://www.youtube.com/watch?v=G06dEcZ-QTg https://arxiv.org/abs/1710.10196

StyleGAN: https://arxiv.org/abs/1812.04948 https://machinelearningmastery.com/introduction-to-style-generative-adversarial-network-stylegan/

---Copyright Info--- Music is from https://filmmusic.io , intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)