Machine Learning Street Talk (MLST)

#55 Self-Supervised Vision Models (Dr. Ishan Misra - FAIR).


Listen Later

Dr. Ishan Misra is a Research Scientist at Facebook AI Research where he works on Computer Vision and Machine Learning. His main research interest is reducing the need for human supervision, and indeed, human knowledge in visual learning systems. He finished his PhD at the Robotics Institute at Carnegie Mellon. He has done stints at Microsoft Research, INRIA and Yale. His bachelors is in computer science where he achieved the highest GPA in his cohort. 


Ishan is fast becoming a prolific scientist, already with more than 3000 citations under his belt and co-authoring with Yann LeCun; the godfather of deep learning.  Today though we will be focusing an exciting cluster of recent papers around unsupervised representation learning for computer vision released from FAIR. These are; DINO: Emerging Properties in Self-Supervised Vision Transformers, BARLOW TWINS: Self-Supervised Learning via Redundancy Reduction and PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with

Support Samples. All of these papers are hot off the press, just being officially released in the last month or so. Many of you will remember PIRL: Self-Supervised Learning of Pretext-Invariant Representations which Ishan was the primary author of in 2019.


References;


Shuffle and Learn - https://arxiv.org/abs/1603.08561

DepthContrast - https://arxiv.org/abs/2101.02691

DINO - https://arxiv.org/abs/2104.14294

Barlow Twins - https://arxiv.org/abs/2103.03230

SwAV - https://arxiv.org/abs/2006.09882

PIRL - https://arxiv.org/abs/1912.01991

AVID - https://arxiv.org/abs/2004.12943 (best paper candidate at CVPR'21 (just announced over the weekend) - http://cvpr2021.thecvf.com/node/290)

 

Alexei (Alyosha) Efros

http://people.eecs.berkeley.edu/~efros/

http://www.cs.cmu.edu/~tmalisie/projects/nips09/

 

Exemplar networks

https://arxiv.org/abs/1406.6909

 

The bitter lesson - Rich Sutton

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

 

Machine Teaching: A New Paradigm for Building Machine Learning Systems

https://arxiv.org/abs/1707.06742

 

POET

https://arxiv.org/pdf/1901.01753.pdf

...more
View all episodesView all episodes
Download on the App Store

Machine Learning Street Talk (MLST)By Machine Learning Street Talk (MLST)

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

95 ratings


More shows like Machine Learning Street Talk (MLST)

View all
The a16z Show by Andreessen Horowitz

The a16z Show

1,105 Listeners

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

442 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

305 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

343 Listeners

Practical AI by Practical AI LLC

Practical AI

209 Listeners

Google DeepMind: The Podcast by Hannah Fry

Google DeepMind: The Podcast

205 Listeners

Last Week in AI by Skynet Today

Last Week in AI

314 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

551 Listeners

Big Technology Podcast by Alex Kantrowitz

Big Technology Podcast

513 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

148 Listeners

Latent Space: The AI Engineer Podcast by Latent.Space

Latent Space: The AI Engineer Podcast

101 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

228 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

688 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

475 Listeners

AI + a16z by a16z

AI + a16z

34 Listeners