Brain Inspired

Matt and I discuss how cognition and behavior drifts over the course of minutes and hours, and how global brain activity drifts with it. How does the brain continue to produce steady perception and action in the midst of such drift? We also talk about how to think about variability in neural activity. How much of it is noise and how much of it is hidden important activity? Finally, we discuss the effect of recording more and more neurons simultaneously, collecting bigger and bigger datasets, plus guest questions from Adam Snyder and Patrick Mayo.

• Smith Lab.
• Twitter: @SmithLabNeuro.
• Related:
  • Slow drift of neural activity as a signature of impulsivity in macaque visual and prefrontal cortex.
• Artwork by Melissa Neely

Take home points:

• The “noise” in the variability of neural activity is likely just activity devoted to processing other things.
• Recording lots of neurons simultaneously helps resolve the question of what’s noise and how much information is in a population of neurons.
• There’s a neural signature of the behavioral “slow drift” of our internal cognitive state.
• The neural signature is global, and it’s an open question how the brain compensates to produce steady perception and action.

Timestamps:

0:00 - Intro

Randy and I discuss his LEABRA cognitive architecture that aims to simulate the human brain, plus his current theory about how a loop between cortical regions and the thalamus could implement predictive learning and thus solve how we learn with so few examples. We also discuss what Randy thinks is the next big thing neuroscience can contribute to AI (thanks to a guest question from Anna Schapiro), and much more.

• Computational Cognitive Neuroscience Laboratory.
• The papers we discuss or mention:
  • The Leabra Cognitive Architecture: How to Play 20 Principles with Nature and Win!
  • Deep Predictive Learning in Neocortex and Pulvinar.
  • Unraveling the Mysteries of Motivation.
• His youTube series detailing the theory and workings of Leabra:
  • Computational Cognitive Neuroscience.
• The free textbook:
  • Computational Cognitive Neuroscience

A few take-home points:

• Leabra has been a slow incremental project, inspired in part by Alan Newell’s suggested approach.
• Randy began by developing a learning algorithm that incorporated both kinds of biological learning (error-driven and associative).
• Leabra's core is 3 brain areas - frontal cortex, parietal cortex, and hippocampus - and has grown from there.
• There’s a constant balance between biological realism and computational feasibility.
• It’s important that a cognitive architecture address multiple levels- micro-scale, macro-scale, mechanisms, functions, and so on.
• Deep predictive learning is a possible brain mechanism whereby predictions from higher layer cortex precede input from lower layer cortex in the thalamus, where an error is computed and used to drive learning.
• Randy believes our metacognitive ability to know what we do and don’t know is a key next function to build into AI.

Timestamps:

When a waiter hands me the bill, how do I know whether to pay it myself or let my date pay? On this episode, I get a progress update from Dileep on his company, Vicarious, since Dileep's last episode. We also talk broadly about his experience running Vicarious to develop AGI and robotics. Then we turn to his latest brain-inspired AI efforts using cloned structured probabilistic graph models to develop an account of how the hippocampus makes a model of the world and represents our cognitive maps in different contexts, so we can simulate possible outcomes to choose how to act.

Special guest questions from Brad Love (episode 70: How We Learn Concepts) .

• Vicarious website - Dileep's AGI robotics company.
• Twitter: @dileeplearning.
• Papers we discuss:
  • Learning cognitive maps as structured graphs for vicarious evaluation.
  • A detailed mathematical theory of thalamic and cortical microcircuits based on inference in a generative vision model.
• Probabilistic graphical models.
• Hierarchical temporal memory.

Time stamps:

0:00 - Intro

Ken and I discuss open-endedness, the pursuit of ambitious goals by seeking novelty and interesting products instead of advancing directly toward defined objectives. We talk about evolution as a prime example of an open-ended system that has produced astounding organisms, Ken relates how open-endedness could help advance artificial intelligence and neuroscience, and we discuss a range of topics related to the general concept of open-endedness, and Ken takes a couple questions from Stefan Leijnen and Melanie Mitchell.

Related:

• Ken's website.
• Twitter: @kenneth0stanley.
• The book:
  • Why Greatness Cannot Be Planned: The Myth of the Objective by Kenneth Stanley and Joel Lehman.
• Papers:
  • Evolving Neural Networks Through Augmenting Topologies (2002)
  • Minimal Criterion Coevolution: A New Approach to Open-Ended Search

Some key take-aways:

• Many of the best inventions were not the result of trying to achieve a specific objective.
• Open-endedness is the pursuit of ambitious advances without a clearly defined objective.
• Evolution is a quintessential example of an open-ended process: it produces a vast array of complex beings by searching the space of possible organisms, constrained by the environment, survival, and reproduction.
• Perhaps the key to developing artificial general intelligence is by following an open-ended path rather that pursing objectives (solving the same old benchmark tasks, etc.).

0:00 - Intro

Ida and I discuss the current landscape of reinforcement learning in both natural and artificial intelligence, and how the old story of two RL systems in brains - model-free and model-based - is giving way to a more nuanced story of these two systems constantly interacting and additional RL strategies between model-free and model-based to drive the vast repertoire of our habits and goal-directed behaviors. We discuss Ida’s work on one of those “in-between” strategies, the successor representation RL strategy, which maps onto brain activity and accounts for behavior. We also discuss her interesting background and how it affects her outlook and research pursuit, and the role philosophy has played and continues to play in her thought processes.

Related links:

• Ida’s website.
• Twitter: @criticalneuro.
• A nice review of what we discuss:
  • Learning Structures: Predictive Representations, Replay, and Generalization.

Time stamps:

0:00 - Intro

David, Gyuri, and I discuss the issues they argue for in their back and forth commentaries about the importance of neuroscience and psychology, or implementation-level and computational-level, to advance our understanding of brains and minds - and the names we give to the things we study. Gyuri believes it’s time we use what we know and discover about brain mechanisms to better describe the psychological concepts we refer to as explanations for minds; David believes the psychological concepts are constantly being refined and are just as valid as objects of study to understand minds. They both agree these are important and enjoyable topics to debate.

Related:

• Buzsáki lab; Poeppel lab
• Twitter: @davidpoeppel.
• The papers we discuss or mention:
  • Calling Names by Christophe Bernard
  • The Brain–Cognitive Behavior Problem: A Retrospective by György Buzsáki.
  • Against the Epistemological Primacy of the Hardware: The Brain from Inside Out, Turned Upside Down by David Poeppel.
• Books:
  • The Brain from Inside Out by György Buzsáki.
  • The Cognitive Neurosciences (edited by David Poeppel et al).

Timeline:

0:00 - Intro

Jane and I discuss the relationship between AI and neuroscience (cognitive science, etc), from her perspective at Deepmind after a career researching natural intelligence. We also talk about her meta-reinforcement learning work that connects deep reinforcement learning with known brain circuitry and processes, and finally we talk about her recent work using evolutionary strategies to develop altruism and cooperation among the agents in a multi-agent reinforcement learning environment.

Related:

• Jane’s website.
• Twitter: @janexwang. 
• The papers we discuss or mention:
  • Learning to reinforcement learn.
  • Blog post with a link to the paper: Prefrontal cortex as a meta-reinforcement learning system.
  • Deep Reinforcement Learning and its Neuroscientific Implications
  • Evolving Intrinsic Motivations for Altruistic Behavior.
• Books she recommended:
  • Human Compatible: AI and the Problem of Control, by Stuart Russell:
  • Algorithms to Live By, by Brian Christian and Tom Griffiths.

Timeline:

0:00 - Intro

Steve and I discuss his long and productive career as a theoretical neuroscientist. We cover his tried and true method of taking a large body of psychological behavioral findings, determining how they fit together and what’s paradoxical about them, developing design principles, theories, and models from that body of data, and using experimental neuroscience to inform and confirm his model predictions. We talk about his Adaptive Resonance Theory (ART) to describe how our brains are self-organizing, adaptive, and deal with changing environments. We also talk about his complementary computing paradigm to describe how two systems can complement each other to create emergent properties neither system can create on its own , how the resonant states in ART support consciousness, his place in the history of both neuroscience and AI, and quite a bit more.

Related:

• Steve's BU website.
• Some papers we discuss or mention (much more on his website):
  • Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world.
  • Towards solving the Hard Problem of Consciousness: The varieties of brain resonances and the conscious experiences that they support.
  • A Path Toward Explainable AI and Autonomous Adaptive Intelligence: Deep Learning, Adaptive Resonance, and Models of Perception, Emotion, and Action.

Topics Time stamps:

0:00 - Intro

Pieter and I discuss his ongoing quest to figure out how the brain implements learning that solves the credit assignment problem, like backpropagation does for neural networks. We also talk about his work to understand how we perceive individual objects in a crowded scene, his neurophysiological recordings in support of the global neuronal workspace hypothesis of consciousness, and the visual prosthetic device he’s developing to cure blindness by directly stimulating early visual cortex. 

Related:

• Pieter's lab website.
• Twitter: @Pieters_Tweet.
• His startup to cure blindness: Phosphoenix.
• Talk:
  • Seeing and thinking with your visual brain
• The papers we discuss or mention:
  • Control of synaptic plasticity in deep cortical networks.
  • A Biologically Plausible Learning Rule for Deep Learning in the Brain.
  • Conscious Processing and the Global Neuronal Workspace Hypothesis.
• Pieter's neuro-origin book inspiration (like so many others): Gödel, Escher, Bach: An Eternal Golden Braid by Douglas Hofstadter.

Daeyeol and I discuss his book Birth of Intelligence: From RNA to Artificial Intelligence, which argues intelligence is a function of and inseparable from life, bound by self-replication and evolution. The book covers a ton of neuroscience related to decision making and learning, though we focused on a few theoretical frameworks and ideas like division of labor and principal-agent relationships to understand how our brains and minds are related to our genes, how AI is related to humans (for now), metacognition, consciousness, and a ton more.

Related:

• Lee Lab for Learning and Decision Making.
• Twitter: @daeyeol_lee.
• Daeyeol’s side passion, creating music.
• His book: Birth of Intelligence: From RNA to Artificial Intelligence.