This episode breaks down the 'Relational Recurrent Neural Networks' paper, which proposes a novel neural network architecture, the Relational Memory Core (RMC), designed to enhance relational reasoning in recurrent neural networks. The RMC utilizes multi-head dot product attention to enable interactions between memory slots, facilitating a more sophisticated understanding of the relationships between stored information. The researchers demonstrate the efficacy of the RMC across various tasks, including a toy problem explicitly designed to assess relational reasoning, program evaluation, reinforcement learning, and language modelling. The paper argues that explicit memory interaction mechanisms are crucial for complex tasks requiring relational reasoning, and the RMC showcases a significant improvement in performance over traditional recurrent models.

Audio : (Spotify) https://open.spotify.com/episode/1Kns0vUoZUv9YnsXym7yMQ?si=-_vaHn7uTJi5SttnjmBQYw

Paper: https://arxiv.org/pdf/1806.01822