TalkRL: The Reinforcement Learning Podcast

Saikrishna Gottipati is an RL Researcher at AI Redefined, working on RL, MARL, human in the loop learning.

Featured References

Cogment: Open Source Framework For Distributed Multi-actor Training, Deployment & Operations
AI Redefined, Sai Krishna Gottipati, Sagar Kurandwad, Clodéric Mars, Gregory Szriftgiser, François Chabot

Do As You Teach: A Multi-Teacher Approach to Self-Play in Deep Reinforcement Learning
Currently under review

Learning to navigate the synthetically accessible chemical space using reinforcement learning
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Karam J. Thomas, Simon Blackburn, Connor W. Coley, Jian Tang, Sarath Chandar, Yoshua Bengio

Additional References

• Asymmetric self-play for automatic goal discovery in robotic manipulation, 2021 OpenAI et al 
• Continuous Coordination As a Realistic Scenario for Lifelong Learning, 2021 Nekoei et al

Episode sponsor: Anyscale

Ray Summit 2022 is coming to San Francisco on August 23-24.
Hear how teams at Dow, Verizon, Riot Games, and more are solving their RL challenges with Ray's RLlib.

Register at raysummit.org and use code RAYSUMMIT22RL for a further 25% off the already reduced prices.

Aravind Srinivas is back!  He is now a research Scientist at OpenAI.

Featured References

Decision Transformer: Reinforcement Learning via Sequence Modeling
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch

VideoGPT: Video Generation using VQ-VAE and Transformers
Wilson Yan, Yunzhi Zhang, Pieter Abbeel, Aravind Srinivas

Dr. Rohin Shah is a Research Scientist at DeepMind, and the editor and main contributor of the Alignment Newsletter.

Featured References

The MineRL BASALT Competition on Learning from Human Feedback
Rohin Shah, Cody Wild, Steven H. Wang, Neel Alex, Brandon Houghton, William Guss, Sharada Mohanty, Anssi Kanervisto, Stephanie Milani, Nicholay Topin, Pieter Abbeel, Stuart Russell, Anca Dragan

Preferences Implicit in the State of the World
Rohin Shah, Dmitrii Krasheninnikov, Jordan Alexander, Pieter Abbeel, Anca Dragan

Benefits of Assistance over Reward Learning
Rohin Shah, Pedro Freire, Neel Alex, Rachel Freedman, Dmitrii Krasheninnikov, Lawrence Chan, Michael D Dennis, Pieter Abbeel, Anca Dragan, Stuart Russell

On the Utility of Learning about Humans for Human-AI Coordination
Micah Carroll, Rohin Shah, Mark K. Ho, Thomas L. Griffiths, Sanjit A. Seshia, Pieter Abbeel, Anca Dragan

Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, A. D. Dragan, Rohin Shah

Additional References

• AGI Safety Fundamentals, EA Cambridge

Jordan Terry is a PhD candidate at University of Maryland, the maintainer of Gym, the maintainer and creator of PettingZoo and the founder of Swarm Labs.

Featured References

PettingZoo: Gym for Multi-Agent Reinforcement Learning
J. K. Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis Santos, Rodrigo Perez, Caroline Horsch, Clemens Dieffendahl, Niall L. Williams, Yashas Lokesh, Praveen Ravi

PettingZoo on Github

gym on Github

Additional References

• Time Limits in Reinforcement Learning, Pardo et al 2017
• Deep Reinforcement Learning at the Edge of the Statistical Precipice, Agarwal et al 2021

Robert Tjarko Lange is a PhD student working at the Technical University Berlin.

Featured References

Learning not to learn: Nature versus nurture in silico
Lange, R. T., & Sprekeler, H. (2020)

On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning
Vischer, M. A., Lange, R. T., & Sprekeler, H. (2021).

Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions
Lange, R. T., & Faisal, A. (2019).

MLE-Infrastructure on Github


Additional References

• RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning, Duan et al 2016
• Learning to reinforcement learn, Wang et al 2016
• Decision Transformer: Reinforcement Learning via Sequence Modeling, Chen et al 2021

We hear about the idea of PERLS and why its important to talk about.

• Political Economy of Reinforcement Learning (PERLS) Workshop at NeurIPS 2021 on Tues Dec 14th 
• NeurIPS 2021

Amy Zhang is a postdoctoral scholar at UC Berkeley and a research scientist at Facebook AI Research. She will be starting as an assistant professor at UT Austin in Spring 2023. 

Featured References 

Invariant Causal Prediction for Block MDPs 
Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup 

Multi-Task Reinforcement Learning with Context-based Representations 
Shagun Sodhani, Amy Zhang, Joelle Pineau 

MBRL-Lib: A Modular Library for Model-based Reinforcement Learning 
Luis Pineda, Brandon Amos, Amy Zhang, Nathan O. Lambert, Roberto Calandra 

Additional References 

• Amy Zhang - Exploring Context for Better Generalization in Reinforcement Learning @ UCL DARK 
• ICML 2020 Poster session: Invariant Causal Prediction for Block MDPs 
• Clare Lyle - Invariant Prediction for Generalization in Reinforcement Learning @ Simons Institute 

Xianyuan Zhan is currently a research assistant professor at the Institute for AI Industry Research (AIR), Tsinghua University.  He received his Ph.D. degree at Purdue University. Before joining Tsinghua University, Dr. Zhan worked as a researcher at Microsoft Research Asia (MSRA) and a data scientist at JD Technology.  At JD Technology, he led the research that uses offline RL to optimize real-world industrial systems. 

Featured References 

DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning
Xianyuan Zhan, Haoran Xu, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng 

Eugene Vinitsky is a PhD student at UC Berkeley advised by Alexandre Bayen. He has interned at Tesla and Deepmind.  

Featured References 

A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings 
Eugene Vinitsky, Raphael Köster, John P. Agapiou, Edgar Duéñez-Guzmán, Alexander Sasha Vezhnevets, Joel Z. Leibo 

Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL 
Eugene Vinitsky, Nathan Lichtle, Kanaad Parvate, Alexandre Bayen 

Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion 
Eugene Vinitsky; Kanaad Parvate; Aboudy Kreidieh; Cathy Wu; Alexandre Bayen 2018 

The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games 
Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, Yi Wu 

Additional References 

• SUMO: Simulation of Urban MObility 

Dr. Jess Whittlestone is a Senior Research Fellow at the Centre for the Study of Existential Risk and the Leverhulme Centre for the Future of Intelligence, both at the University of Cambridge. 

Featured References 

The Societal Implications of Deep Reinforcement Learning 
Jess Whittlestone, Kai Arulkumaran, Matthew Crosby 

Artificial Canaries: Early Warning Signs for Anticipatory and Democratic Governance of AI 
Carla Zoe Cremer, Jess Whittlestone 


Additional References 

• CogX: Cutting Edge: Understanding AI systems for a better AI policy, featuring Jack Clark and Jess Whittlestone