This is part 2 of my conversation with Brad (listen to part 1 here). We discuss how Moore’s law is on its last legs, and his ideas for how neuroscience - in particular neural algorithms - may help computing continue to scale in a post-Moore’s law world. We also discuss neuromporphics in general, and more.

• Brad's homepage.
• Follow Brad on Twitter: @jbimaknee.
• The paper we discuss:
  • Neural Algorithms and Computing Beyond Moore's Law.
• Check out the Neuro Inspired Computing Elements (NICE) workshop - lots of great talks and panel discussions.

In this first part of our discussion, Brad and I discuss the state of neuromorphics and its relation to neuroscience and artificial intelligence.  He describes his work adding new neurons to deep learning networks during training, called neurogenesis deep learning, inspired by how neurogenesis in the dentate gyrus of the hippocampus helps learn new things while keeping previous memories intact. We also talk about his method to transform deep learning networks into spiking neural networks so they can run on neuromorphic hardware, and the neuromorphics workshop he puts on every year, the Neuro Inspired Computational Elements (NICE) workshop.

Show Notes:

• Brad's homepage.
• Follow Brad on Twitter: @jbimaknee.
• The papers we discuss:
  • Computational Influence of Adult Neurogenesis on Memory Encoding.
  • Neurogenesis Deep Learning.
  • Training deep neural networks for binary communication with the Whetstone method.
    • And here's the arXiv version.
• Check out the Neuro Inspired Computing Elements (NICE) workshop - lots of great talks and panel discussions.

Show Notes:

• Nando’s CIFAR page.
• Follow Nando on Twitter: @NandoDF
• He's giving a keynote address at Cognitive Computational Neuroscience Meeting 2020.
• Check out his famous machine learning lectures on Youtube.
• Papers we (more allude to than) discuss:
  • Neural Programmer-Interpreters.
  • Learning to learn by gradient descent by gradient descent.
  • Dueling Network Architectures for Deep Reinforcement Learning.
  • Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions.
  • One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL.

Show notes:

• Check out Anne's lab website.
• Follow her on twitter: @anne_churchland
• Anne's List, the list of female systems neuroscientists to invite as speakers.
• The papers we discuss:
  • Single-trial neural dynamics are dominated by richly varied movements.
  • Lapses in perceptual judgments reflect exploration.
  • Complexity vs Stimulus-Response Compatibility vs Stimulus-response ethological validity.
  • Perceptual Decision-Making: A Field in the Midst of a Transformation.

Show notes:

• Máté's Cambridge website.
• He's part of the Computational Learning and Memory Group there.
• Here's his webpage at Central European University.
• A review to introduce his subsequent work:
  • Statistically optimal perception and learning: from behavior to neural representations.
• Related recent talks:
  • Bayesian models of perception, cognition and learning - CCCN 2017.
  • Sampling: coding, dynamics, and computation in the cortex (Cosyne 2018).

Show notes:

• Nathaniel will deliver a keynote address at the upcoming CCN conference.
• Check out his lab website.
• Follow him on Twitter: @nathanieldaw.
• The paper we discuss:
  • Prioritized memory access explains planning and hippocampal replay
  • Or see a related talk: Rational planning using prioritized experience replay.

Show notes:

• Roshan will deliver a keynote address at the upcoming CCN conference.
• Roshan's Motivational and Cognitive Control lab.
• Follow her on Twitter: @CoolsControl.
• Her TED Talk on Trusting Science.
• Papers related to the research we discuss:
  • The costs and benefits of brain dopamine for cognitive control.
  • Or see her variety of related works.

Show notes:

• This is the first in a series of episodes where I interview keynote speakers at the upcoming Cognitive Computational Neuroscience conference in September in Berlin.
  • Thomas Naseralis summarizes the origins and vision of the CCN.
• Tim’s Neuroscience homepage:
• The papers we discuss:
  • Generalisation of structural knowledge in the hippocampal-entorhinal system (referred to in the podcast at "The Tolman Eichenbaum Machine”)
• Human replay spontaneously reorganizes experience. (In press at Cell - below is an abstract for it from COSYNE 2018)
  • Inference in replay through factorized representations.

Show notes:

• Tony’s lab site, where there are links to his auditory decision making work and connectome work we discuss.
• Here are a few talks online about that:
  • Corticostriatal circuits underlying auditory decisions.

Show Notes:

• Federico's website.
• Federico’s papers we discuss:
  • Conflicting emergences. Weak vs. strong emergence for the modelling of brain function
  • From homeostasis to behavior: balanced activity in an exploration of embodied dynamic environmental-neural interaction
• Free Energy Principle.
• Integrated Information Theory.
• The Tononi paper about Integrated Information Theory and its relation to emergence:
  • Quantifying causal emergence shows that macro can beat micro
• Default mode as large scale oscillation:
  • The brain's code and its canonical computational motifs. From sensory cortex to the default mode network: A multi-scale model of brain function in health and disease.