Machine Learning Street Talk (MLST)

#030 Multi-Armed Bandits and Pure-Exploration (Wouter M. Koolen)


Listen Later

This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher discuss multi-arm bandits and pure exploration with Dr. Wouter M. Koolen, Senior Researcher, Machine Learning group, Centrum Wiskunde & Informatica.


Wouter specialises in machine learning theory, game theory, information theory, statistics and optimisation. Wouter is currently interested in pure exploration in multi-armed bandit models, game tree search, and accelerated learning in sequential decision problems. His research has been cited 1000 times, and he has been published in NeurIPS, the number 1 ML conference 14 times as well as lots of other exciting publications.


Today we are going to talk about two of the most studied settings in control, decision theory, and learning in unknown environment which are the multi-armed bandit (MAB) and reinforcement learning (RL) approaches

- when can an agent stop learning and start exploiting using the knowledge it obtained

- which strategy leads to minimal learning time


00:00:00 What are multi-arm bandits/show trailer

00:12:55 Show introduction

00:15:50 Bandits 

00:18:58 Taxonomy of decision framework approaches 

00:25:46 Exploration vs Exploitation 

00:31:43 the sharp divide between modes 

00:34:12 bandit measures of success 

00:36:44 connections to reinforcement learning 

00:44:00 when to apply pure exploration in games 

00:45:54 bandit lower bounds, a pure exploration renaissance 

00:50:21 pure exploration compiler dreams 

00:51:56 what would the PX-compiler DSL look like 

00:57:13 the long arms of the bandit 

01:00:21 causal models behind the curtain of arms 

01:02:43 adversarial bandits, arms trying to beat you 

01:05:12 bandits as an optimization problem 

01:11:39 asymptotic optimality vs practical performance 

01:15:38 pitfalls hiding under asymptotic cover 

01:18:50 adding features to bandits 

01:27:24 moderate confidence regimes  

01:30:33 algorithms choice is highly sensitive to bounds 

01:46:09 Post script: Keith interesting piece on n quantum 


http://wouterkoolen.info

https://www.cwi.nl/research-groups/ma...


#machinelearning

...more
View all episodesView all episodes
Download on the App Store

Machine Learning Street Talk (MLST)By Machine Learning Street Talk (MLST)

  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

84 ratings


More shows like Machine Learning Street Talk (MLST)

View all
Data Skeptic by Kyle Polich

Data Skeptic

481 Listeners

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

440 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

298 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

323 Listeners

Machine Learning Guide by OCDevel

Machine Learning Guide

765 Listeners

Practical AI by Practical AI LLC

Practical AI

189 Listeners

ManifoldOne by Steve Hsu

ManifoldOne

87 Listeners

Google DeepMind: The Podcast by Hannah Fry

Google DeepMind: The Podcast

199 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

372 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

122 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

199 Listeners

Unsupervised Learning by by Redpoint Ventures

Unsupervised Learning

40 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

76 Listeners

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

441 Listeners

Training Data by Sequoia Capital

Training Data

36 Listeners