Talking Papers Podcast

Talking Papers Podcast

By Itzik Ben-ShabatScienceTechnologyEducation
Download on the App Store

All Talking Papers Podcast episodes

  • Yicong Hong - VLN BERT

    PAPER TITLE:
    "VLN BERT:  A Recurrent Vision-and-Language BERT for Navigation"

    AUTHORS: 
    Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, Stephen Gould

    ABSTRACT:
    Accuracy of many visiolinguistic tasks has benefited significantly from the application of vision-and-language (V&L) BERT. However, its application for the task of vision and-language navigation (VLN) remains limited. One reason for this is the difficulty adapting the BERT architecture to the partially observable Markov decision process present in VLN, requiring history-dependent attention and decision making. In this paper we propose a recurrent BERT model that is time-aware for use in VLN. Specifically, we equip the BERT model with a recurrent function that maintains cross-modal state information for the agent. Through extensive experiments on R2R and REVERIE we demonstrate that our model can replace more complex encoder-decoder models to achieve state-of-the-art results. Moreover, our approach can be generalised to other transformer-based architectures, supports pre-training, and is capable of solving navigation and referring expression tasks simultaneously.

    CODE:
    💻  https://github.com/YicongHong/Recurrent-VLN-BERT

    LINKS AND RESOURCES
    👱Yicong's page

    RELATED PAPERS:
    📚 Attention is All You Need
    📚 Towards learning a generic agent for vision-and-language navigation via pre-training

    CONTACT:
    -----------------
    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]

    This episode was recorded on April, 16th 2021.

    SUBSCRIBE AND FOLLOW:
    🎧Subscribe on your favourite podcast app:  https://talking.papers.podcast.itzikbs.com
    📧Subscribe to our mailing list: http://eepurl.com/hRznqb
    🐦Follow us on Twitter: https://twitter.com/talking_papers
    🎥YouTube Channel: https://bit.ly/3eQOgwP

    #talkingpapers #CVPR2021 #VLNBERT
    #VLN #VisionAndLanguageNavigation #VisionAndLanguage #machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence 

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    23 min
  • Despoina Paschalidou - Neural Parts

    PAPER TITLE
     Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks

    AUTHORS

    Despoina Paschalidou , Angelos Katharopoulos, Andreas Geiger, Sanja Fidler


    ABSTRACT

    Impressive progress in 3D shape extraction led to representations that can capture object geometries with high fidelity. In parallel, primitive-based methods seek to represent objects as semantically consistent part arrangements. However, due to the simplicity of existing primitive representations, these methods fail to accurately reconstruct 3D shapes using a small number of primitives/parts. We address the trade-off between reconstruction quality and number of parts with Neural Parts, a novel 3D primitive representation that defines primitives using an Invertible Neural Network (INN) which implements homeomorphic mappings between a sphere and the target object. The INN allows us to compute the inverse mapping of the homomorphism, which in turn, enables the efficient computation of both the implicit surface function of a primitive and its mesh, without any additional post-processing. Our model learns to parse 3D objects into semantically consistent part arrangements without any part-level supervision. Evaluations on ShapeNet, D-FAUST and FreiHAND demonstrate that our primitives can capture complex geometries and thus simultaneously achieve geometrically accurate as well as interpretable reconstructions using an order of magnitude fewer primitives than state-of-the-art shape abstraction methods.

    RELATED PAPERS

    📚 "KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control"

    📚 "Learning Shape Abstractions by Assembling Volumetric Primitives": Volumetric primitives"

    📚 "Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids"

    📚 "CvxNet: Learnable Convex Decomposition"
    📚 "Neural Star Domain as Primitive Representation"


    LINKS AND RESOURCES

    💻 Project Page: https://paschalidoud.github.io/neural_parts

    💻 CODE: https://github.com/paschalidoud/neural_parts

    💻Blog Post


     CONTACT

    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]


    SUBSCRIBE AND FOLLOW

     🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

     📧Subscribe to our mailing list: http://eepurl.com/hRznqb

     🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    This episode was recorded on April, 25th 2021.

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    41 min
  • Guy Gafni - NerFACE

    PAPER TITLE:
    Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction

    AUTHORS: 
    Guy Gafni      Justus Thies      Michael Zollhöfer     Matthias Nießner   

    Project page: https://gafniguy.github.io/4D-Facial-Avatars/

    CODE:
    💻https://github.com/gafniguy/4D-Facial-Avatars

    ABSTRACT:
    We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or head-poses is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photo-realistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.


    RELATED PAPERS:
    📚Representing Scenes as Neural Radiance Fields for View Synthesis
    📚Deep Video Portraits
    📚Nerfies: Deformable Neural Radiance Fields
    📚AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis

    CONTACT:
    -----------------
    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]

    TIME STAMPS
    -----------------------

    00:00   
    00:07 Intro
    00:27 Authors
    01:16 Abstract / TLDR
    02:54 Motivation
    12:24 Related Work
    13:20 Approach
    17:10 Results
    27:05 Conclusions and future work
    32:12 Outro

    #talkingpapers #CVPR2021 #NeRF
    #machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence #FacialAvatars

    Recorded on April, 2nd 2021.

    SUBSCRIBE AND FOLLOW:
    🎧Subscribe on your favourite podcast app:  https://talking.papers.podcast.itzikbs.com
    📧Subscribe to our mailing list: http://eepurl.com/hRznqb
    🐦Follow us on Twitter: https://twitter.com/talking_papers
    🎥YouTube Channel: https://bit.ly/3eQOgwP

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    33 min
  • Jing Zhang - UC-Net

    PAPER TITLE:
    "UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders"

    AUTHORS:
    Jing Zhang, Deng-Ping Fan, Yuchao Dai, Saeed Anwar, Fatemeh Sadat Saleh, Tong Zhang, Nick Barnes

    ABSTRACT:
    In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection methods treat the saliency detection task as a point estimation problem, and produce a single saliency map following a deterministic learning pipeline. Inspired by the saliency data labeling process, we propose probabilistic RGB-D saliency detection network via conditional variational autoencoders to model human annotation uncertainty and generate multiple saliency maps for each input image by sampling in the latent space. With the proposed saliency consensus process, we are able to generate an accurate saliency map based on these multiple predictions. Quantitative and qualitative evaluations on six challenging benchmark datasets against 18 competing algorithms demonstrate the effectiveness of our approach in learning the distribution of saliency maps, leading to a new state-of-the-art in RGB-D saliency detection.

    💻SUBSCRIBE AND FOLLOW:
    🎧Subscribe on your favourite podcast app:  https://talking.papers.podcast.itzikbs.com
    📧Subscribe to our mailing list: http://eepurl.com/hRznqb
    🐦Follow us on Twitter: https://twitter.com/talking_papers
    🎥YouTube Channel: https://bit.ly/3eQOgwP

    CODE:
    💻https://github.com/JingZhang617/UCNet

    RELATED PAPERS:
    📚A probabilistic u-net for segmentation of ambiguous images
    📚Learning structured output representation using deep conditional generative models

    CONTACT:
    -----------------
    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]

    TIME STAMPS
    -----------------------
    00:00 |  
    00:02 |  Intro
    00:31 |  The Authors
    01:07 |  Abstract / TLDR
    02:41 |  Motivation
    07:18 |  Related Work
    09:20 |  Approach
    18:32 |  Results
    24:04 |  Conclusions and future work
    25:42 |  What did reviewer 2 say?
    29:49 |  Outro

    #talkingpapers #CVPR2020 #RGBDSaliency
    #machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    31 min
  • Dylan Campbell - Deep Declarative Networks

    PAPER TITLE:
    "Deep Declarative Networks: a new hope"

    AUTHORS:
    Stephen Gould, Richard Hartley, Dylan Campbell

    ABSTRACT:
    We explore a new class of end-to-end learnable models wherein data processing nodes (or network layers) are defined in terms of desired behaviour rather than an explicit forward function. Specifically, the forward function is implicitly defined as the solution to a mathematical optimization problem. Consistent with nomenclature in the programming languages community, we name these models deep declarative networks. Importantly, we show that the class of deep declarative networks subsumes current deep learning models. Moreover, invoking the implicit function theorem, we show how gradients can be back-propagated through many declaratively defined data processing nodes thereby enabling end-to-end learning. We show how these declarative processing nodes can be implemented in the popular PyTorch deep learning software library allowing declarative and imperative nodes to co-exist within the same network. We also provide numerous insights and illustrative examples of declarative nodes and demonstrate their application for image and point cloud classification tasks.

    💻SUBSCRIBE AND FOLLOW:
    🎧Subscribe on your favourite podcast app:  https://talking.papers.podcast.itzikbs.com
    📧Subscribe to our mailing list: http://eepurl.com/hRznqb
    🐦Follow us on Twitter: https://twitter.com/talking_papers
    🎥YouTube Channel: https://bit.ly/3eQOgwP

    TUTORIALS AND WORKSHOPS:
    ECCV 2020 Tutorial
    CVPR 2020 Workshop

    CODE:
    💻Codebase
    💻Jupiter notebooks

    PAPER:
    "Deep Declarative Networks: a new hope" Preprint
    "Deep Declarative Networks"

    RELATED PAPERS:
    📚"On differentiating parameterized argmin and argmax problems with application to bi-level optimization"
    📚"OptNet: Differentiable Optimization as a Layer in Neural Networks" : 

    CONTACT:
    -----------------
    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]

    #talkingpapers #TPAMI2021 #deepdeclarativenetworks
    #machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence 

    Recorded on March, 31th 2021.

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    30 min
  • Cristian Rodriguez-Opazo - DORi

    Paper title:
    "DORi: Discovering Object Relationships for Moment Localization of a Natural Language Query in a Video"

    Authors:  Cristian Rodriguez-Opazo, Edison Marrese-Taylor, Basura Fernando, Hongdong Li, Stephen Gould

    Abstract:
    This paper studies the task of temporal moment localization in a long untrimmed video using natural language query. Given a query sentence, the goal is to determine the start and end of the relevant segment within the video. Our key innovation is to learn a video feature embedding through a language-conditioned message-passing algorithm suitable for temporal moment localization which captures the relationships between humans, objects and activities in the video. These relationships are obtained by a spatial subgraph that contextualized the scene representation using detected objects and human features. Moreover, a temporal sub-graph captures the activities within the video through time. Our method is evaluated on three standard benchmark datasets, and we also introduce YouCook II as a new benchmark for this task. Experiments show our method outperforms state-of-the-art methods on these datasets, confirming the effectiveness of our approach

    RESOURCES
    -----------------
    Cristian's page: https://crodriguezo.github.io/

    Code:
    https://github.com/crodriguezo/DORi

    Related papers:
    "Proposal free temporal moment localization" : https://bit.ly/3EX1qCM
    "Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs" :  https://bit.ly/3zt4aXA

    Subscribe to the podcast:  https://talking.papers.podcast.itzikbs.com
    Subscribe to our mailing list: http://eepurl.com/hRznqb
    Follow us on Twitter: https://twitter.com/talking_papers
    YouTube Channel: https://bit.ly/3eQOgwP

    CONTACT:
    -----------------
    If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [email protected]

    Recorded on March, 26th 2021.

    🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

    📧Subscribe to our mailing list: http://eepurl.com/hRznqb

    🐦Follow us on Twitter: https://twitter.com/talking_papers

    🎥YouTube Channel: https://bit.ly/3eQOgwP

    28 min