Show Notes

• (2:05) Louis went over his childhood as a self-taught programmer and his early days in school as a freelance developer.
• (4:22) Louis described his overall undergraduate experience getting a Bachelor’s degree in IT Systems Engineering from Hasso Plattner Institute, a highly-ranked computer science university in Germany.
• (6:10) Louis dissected his Bachelor thesis at HPI called “Differentiable Convolutional Neural Network Architectures for Time Series Classification,” — which addresses the problem of automatically designing architectures for time series classification efficiently, using a regularization technique for ConvNet that enables joint training of network weights and architecture through back-propagation.
• (7:40) Louis provided a brief overview of his publication “Transfer Learning for Speech Recognition on a Budget,” — which explores Automatic Speech Recognition training by model adaptation under constrained GPU memory, throughput, and training data.
• (10:31) Louis described his one-year Master of Research degree in Computational Statistics and Machine Learning at the University College London supervised by David Barber.
• (12:13) Louis unpacked his paper “Modular Networks: Learning to Decompose Neural Computation,” published at NeurIPS 2018 — which proposes a training algorithm that flexibly chooses neural modules based on the processed data.
• (15:13) Louis briefly reviewed his technical report, “Scaling Neural Networks Through Sparsity,” which discusses near-term and long-term solutions to handle sparsity between neural layers.
• (18:30) Louis mentioned his report, “Characteristics of Machine Learning Research with Impact,” which explores questions such as how to measure research impact and what questions the machine learning community should focus on to maximize impact.
• (21:16) Louis explained his report, “Contemporary Challenges in Artificial Intelligence,” which covers lifelong learning, scalability, generalization, self-referential algorithms, and benchmarks.
• (23:16) Louis talked about his motivation to start a blog and discussed his two-part blog series on intelligence theories (part 1 on universal AI and part 2 on active inference).
• (27:46) Louis described his decision to pursue a Ph.D. at the Swiss AI Lab IDSIA in Lugano, Switzerland, where he has been working on Meta Reinforcement Learning agents with Jürgen Schmidhuber.
• (30:06) Louis created a very extensive map of reinforcement learning in 2019 that outlines the goal, methods, and challenges associated with the RL domain.
• (33:50) Louis unpacked his blog post reflecting on his experience at NeurIPS 2018 and providing updates on the AGI roadmap regarding topics such as scalability, continual learning, meta-learning, and benchmarks.
• (37:04) Louis dissected his ICLR 2020 paper “Improving Generalization in Meta Reinforcement Learning using Learned Objectives,” which introduces a novel algorithm called MetaGenRL, inspired by biological evolution.
• (44:03) Louis elaborated on his publication “Meta-Learning Backpropagation And Improving It,” which introduces the Variable Shared Meta-Learning framework that unifies existing meta-learning approaches and demonstrates that simple weight-sharing and sparsity in a network are sufficient to express powerful learning algorithms.
• (51:14) Louis expands on his idea to bootstrap AI that entails how the task, the general meta learner, and the unsupervised objective should interact (proposed at the end of his invited talk at NeurIPS 2020).
• (54:14) Louis shared his advice for individuals who want to make a dent in AI research.
• (56:05) Louis shared his three most useful productivity tips.
• (58:36) Closing segment.

Louis’s Contact Info

• Website
• Twitter
• LinkedIn
• Google Scholar
• GitHub

Mentioned Content

Papers and Reports

• Differentiable Convolutional Neural Network Architectures for Time Series Classification (2017)
• Transfer Learning for Speech Recognition on a Budget (2017)
• Modular Networks: Learning to Decompose Neural Computation (2018)
• Contemporary Challenges in Artificial Intelligence (2018)
• Characteristics of Machine Learning Research with Impact (2018)
• Scaling Neural Networks Through Sparsity (2018)
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives (2019)
• Meta-Learning Backpropagation And Improving It (2020)

Blog Posts

• Theories of Intelligence — Part 1 and Part 2 (July 2018)
• Modular Networks: Learning to Decompose Neural Computation (May 2018)
• How to Make Your ML Research More Impactful (Dec 2018)
• A Map of Reinforcement Learning (Jan 2019)
• NeurIPS 2018, Updates on the AI Roadmap (Jan 2019)
• MetaGenRL: Improving Generalization in Meta Reinforcement Learning (Oct 2019)
• General Meta-Learning and Variable Sharing (Nov 2020)

People

• Jeff Clune (for his push on meta-learning research)
• Kenneth Stanley (for his deep thoughts on open-ended learning)
• Jürgen Schmidhuber (for being a visionary scientist)

Book

• “Grit” (by Angela Duckworth)

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit datacast.substack.com/subscribe