Sign up to save your podcasts
Or
Share Deep Learning in Spiking Neural Networks
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Deep Learning in Spiking Neural Networks
This episode explores how spiking neural networks differ from conventional deep networks by representing information as time-based spikes rather than continuous activations, and why that makes them both appealing and difficult to train. It breaks down the main research directions in the field, including local spike-timing-dependent plasticity, direct gradient-based training of spiking models, and ANN-to-SNN conversion, emphasizing that these approaches solve different problems and should not be treated as interchangeable. The discussion highlights the paper’s core argument that spiking models may help bridge neuroscience, energy-efficient neuromorphic hardware, and competitive machine learning, while also questioning whether claims about progress often blur together engineering wins, biological realism, and benchmark performance. Listeners would find it interesting for its clear explanation of the ANN-SNN gap, the tradeoffs behind biologically inspired AI, and the unresolved question of whether spiking systems can become both practical and scientifically meaningful.
Sources:
1. Deep Learning in Spiking Neural Networks — Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothee Masquelier, Anthony S. Maida, 2018
http://arxiv.org/abs/1804.08150
2. Networks of Spiking Neurons: The Third Generation of Neural Network Models — Wolfgang Maass, 1997
https://scholar.google.com/scholar?q=Networks+of+Spiking+Neurons%3A+The+Third+Generation+of+Neural+Network+Models
3. Deep Learning in Spiking Neural Networks — Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothee Masquelier, Anthony Maida, 2019
https://scholar.google.com/scholar?q=Deep+Learning+in+Spiking+Neural+Networks
4. Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks — Emre O. Neftci, Hesham Mostafa, Friedemann Zenke, 2019
https://scholar.google.com/scholar?q=Surrogate+Gradient+Learning+in+Spiking+Neural+Networks%3A+Bringing+the+Power+of+Gradient-Based+Optimization+to+Spiking+Neural+Networks
5. Backpropagation and the Brain — Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman, Geoffrey Hinton, 2020
https://scholar.google.com/scholar?q=Backpropagation+and+the+Brain
6. Fast-Classifying, High-Accuracy Spiking Deep Networks Through Weight and Threshold Balancing — Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, Michael Pfeiffer, 2015
https://scholar.google.com/scholar?q=Fast-Classifying%2C+High-Accuracy+Spiking+Deep+Networks+Through+Weight+and+Threshold+Balancing
7. Training Deep Spiking Neural Networks Using Backpropagation — Jun Haeng Lee, Tobi Delbruck, Michael Pfeiffer, 2016
https://scholar.google.com/scholar?q=Training+Deep+Spiking+Neural+Networks+Using+Backpropagation
8. Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification — Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, Shih-Chii Liu, 2017
https://scholar.google.com/scholar?q=Conversion+of+Continuous-Valued+Deep+Networks+to+Efficient+Event-Driven+Networks+for+Image+Classification
9. Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks — Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi, 2018
https://scholar.google.com/scholar?q=Spatio-Temporal+Backpropagation+for+Training+High-Performance+Spiking+Neural+Networks
10. SLAYER: Spike Layer Error Reassignment in Time — Sumit Bam Shrestha, Garrick Orchard, 2018
https://scholar.google.com/scholar?q=SLAYER%3A+Spike+Layer+Error+Reassignment+in+Time
11. A spiking neural network with continuous local learning for robust online brain machine interface — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=A+spiking+neural+network+with+continuous+local+learning+for+robust+online+brain+machine+interface
12. Adaptive deep spiking neural network with global-local learning via balanced excitatory and inhibitory mechanism — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Adaptive+deep+spiking+neural+network+with+global-local+learning+via+balanced+excitatory+and+inhibitory+mechanism
13. Delay learning based on temporal coding in spiking neural networks — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Delay+learning+based+on+temporal+coding+in+spiking+neural+networks
14. Temporal-coded spiking neural networks with dynamic firing threshold: Learning with event-driven backpropagation — approx. unknown from snippet, 2021
https://scholar.google.com/scholar?q=Temporal-coded+spiking+neural+networks+with+dynamic+firing+threshold%3A+Learning+with+event-driven+backpropagation
15. Spikingformer: Spike-driven residual learning for transformer-based spiking neural network — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Spikingformer%3A+Spike-driven+residual+learning+for+transformer-based+spiking+neural+network
16. Sstformer: Bridging spiking neural network and memory support transformer for frame-event based recognition — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Sstformer%3A+Bridging+spiking+neural+network+and+memory+support+transformer+for+frame-event+based+recognition
17. TE-Spikformer: Temporal-enhanced spiking neural network with transformer — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=TE-Spikformer%3A+Temporal-enhanced+spiking+neural+network+with+transformer
18. NeuronSpark: A Spiking Neural Network Language Model with Selective State Space Dynamics — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=NeuronSpark%3A+A+Spiking+Neural+Network+Language+Model+with+Selective+State+Space+Dynamics
19. SpikingSSMs: Learning long sequences with sparse and parallel spiking state space models — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=SpikingSSMs%3A+Learning+long+sequences+with+sparse+and+parallel+spiking+state+space+models
20. Delays in Spiking Neural Networks: A State Space Model Approach — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Delays+in+Spiking+Neural+Networks%3A+A+State+Space+Model+Approach
21. AI Post Transformers: Directly Trained Spiking DQNs for Atari — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-20-directly-trained-spiking-dqns-for-atari-947693.mp3
22. AI Post Transformers: SpikingBrain: Brain-Inspired LLMs for Efficient Long-Context Processing — Hal Turing & Dr. Ada Shannon, 2025
https://podcast.do-not-panic.com/episodes/spikingbrain-brain-inspired-llms-for-efficient-long-context-processing/
Interactive Visualization: Deep Learning in Spiking Neural Networks
View all episodes
By mcgrofThis episode explores how spiking neural networks differ from conventional deep networks by representing information as time-based spikes rather than continuous activations, and why that makes them both appealing and difficult to train. It breaks down the main research directions in the field, including local spike-timing-dependent plasticity, direct gradient-based training of spiking models, and ANN-to-SNN conversion, emphasizing that these approaches solve different problems and should not be treated as interchangeable. The discussion highlights the paper’s core argument that spiking models may help bridge neuroscience, energy-efficient neuromorphic hardware, and competitive machine learning, while also questioning whether claims about progress often blur together engineering wins, biological realism, and benchmark performance. Listeners would find it interesting for its clear explanation of the ANN-SNN gap, the tradeoffs behind biologically inspired AI, and the unresolved question of whether spiking systems can become both practical and scientifically meaningful.
Sources:
1. Deep Learning in Spiking Neural Networks — Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothee Masquelier, Anthony S. Maida, 2018
http://arxiv.org/abs/1804.08150
2. Networks of Spiking Neurons: The Third Generation of Neural Network Models — Wolfgang Maass, 1997
https://scholar.google.com/scholar?q=Networks+of+Spiking+Neurons%3A+The+Third+Generation+of+Neural+Network+Models
3. Deep Learning in Spiking Neural Networks — Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothee Masquelier, Anthony Maida, 2019
https://scholar.google.com/scholar?q=Deep+Learning+in+Spiking+Neural+Networks
4. Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks — Emre O. Neftci, Hesham Mostafa, Friedemann Zenke, 2019
https://scholar.google.com/scholar?q=Surrogate+Gradient+Learning+in+Spiking+Neural+Networks%3A+Bringing+the+Power+of+Gradient-Based+Optimization+to+Spiking+Neural+Networks
5. Backpropagation and the Brain — Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman, Geoffrey Hinton, 2020
https://scholar.google.com/scholar?q=Backpropagation+and+the+Brain
6. Fast-Classifying, High-Accuracy Spiking Deep Networks Through Weight and Threshold Balancing — Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, Michael Pfeiffer, 2015
https://scholar.google.com/scholar?q=Fast-Classifying%2C+High-Accuracy+Spiking+Deep+Networks+Through+Weight+and+Threshold+Balancing
7. Training Deep Spiking Neural Networks Using Backpropagation — Jun Haeng Lee, Tobi Delbruck, Michael Pfeiffer, 2016
https://scholar.google.com/scholar?q=Training+Deep+Spiking+Neural+Networks+Using+Backpropagation
8. Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification — Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, Shih-Chii Liu, 2017
https://scholar.google.com/scholar?q=Conversion+of+Continuous-Valued+Deep+Networks+to+Efficient+Event-Driven+Networks+for+Image+Classification
9. Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks — Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi, 2018
https://scholar.google.com/scholar?q=Spatio-Temporal+Backpropagation+for+Training+High-Performance+Spiking+Neural+Networks
10. SLAYER: Spike Layer Error Reassignment in Time — Sumit Bam Shrestha, Garrick Orchard, 2018
https://scholar.google.com/scholar?q=SLAYER%3A+Spike+Layer+Error+Reassignment+in+Time
11. A spiking neural network with continuous local learning for robust online brain machine interface — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=A+spiking+neural+network+with+continuous+local+learning+for+robust+online+brain+machine+interface
12. Adaptive deep spiking neural network with global-local learning via balanced excitatory and inhibitory mechanism — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Adaptive+deep+spiking+neural+network+with+global-local+learning+via+balanced+excitatory+and+inhibitory+mechanism
13. Delay learning based on temporal coding in spiking neural networks — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Delay+learning+based+on+temporal+coding+in+spiking+neural+networks
14. Temporal-coded spiking neural networks with dynamic firing threshold: Learning with event-driven backpropagation — approx. unknown from snippet, 2021
https://scholar.google.com/scholar?q=Temporal-coded+spiking+neural+networks+with+dynamic+firing+threshold%3A+Learning+with+event-driven+backpropagation
15. Spikingformer: Spike-driven residual learning for transformer-based spiking neural network — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Spikingformer%3A+Spike-driven+residual+learning+for+transformer-based+spiking+neural+network
16. Sstformer: Bridging spiking neural network and memory support transformer for frame-event based recognition — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Sstformer%3A+Bridging+spiking+neural+network+and+memory+support+transformer+for+frame-event+based+recognition
17. TE-Spikformer: Temporal-enhanced spiking neural network with transformer — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=TE-Spikformer%3A+Temporal-enhanced+spiking+neural+network+with+transformer
18. NeuronSpark: A Spiking Neural Network Language Model with Selective State Space Dynamics — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=NeuronSpark%3A+A+Spiking+Neural+Network+Language+Model+with+Selective+State+Space+Dynamics
19. SpikingSSMs: Learning long sequences with sparse and parallel spiking state space models — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=SpikingSSMs%3A+Learning+long+sequences+with+sparse+and+parallel+spiking+state+space+models
20. Delays in Spiking Neural Networks: A State Space Model Approach — approx. unknown from snippet, recent
https://scholar.google.com/scholar?q=Delays+in+Spiking+Neural+Networks%3A+A+State+Space+Model+Approach
21. AI Post Transformers: Directly Trained Spiking DQNs for Atari — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-20-directly-trained-spiking-dqns-for-atari-947693.mp3
22. AI Post Transformers: SpikingBrain: Brain-Inspired LLMs for Efficient Long-Context Processing — Hal Turing & Dr. Ada Shannon, 2025
https://podcast.do-not-panic.com/episodes/spikingbrain-brain-inspired-llms-for-efficient-long-context-processing/
Interactive Visualization: Deep Learning in Spiking Neural Networks