Smooth Exact Gradient Descent Learning in Spiking Neural Networks

被引：0

作者：

Klos, Christian ^{[1
]}

Memmesheimer, Raoul-Martin ^{[1
]}

机构：

[1] Univ Bonn, Inst Genet, Neural Network Dynam & Computat, D-53115 Bonn, Germany

来源：

PHYSICAL REVIEW LETTERS | 2025年 / 134卷 / 02期

关键词：

ERROR-BACKPROPAGATION; NEURONS; SIMULATION; SPARSE; CHAOS; MODEL; FIRE;

D O I：

10.1103/PhysRevLett.134.027301

中图分类号：

O4 [物理学];

学科分类号：

0702 ;

摘要：

Gradient descent prevails in artificial neural network training, but seems inept for spiking neural networks as small parameter changes can cause sudden, disruptive appearances and disappearances of spikes. Here, we demonstrate exact gradient descent based on continuously changing spiking dynamics. These are generated by neuron models whose spikes vanish and appear at the end of a trial, where it cannot influence subsequent dynamics. This also enables gradient-based spike addition and removal. We illustrate our scheme with various tasks and setups, including recurrent and deep, initially silent networks.

引用

页数：8

共 50 条

[31] Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
Veeriah, Vivek
Zhang, Shangtong
Sutton, Richard S.
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2017, PT I, 2017, 10534 : 445 - 459
[32] LEARNING SHALLOW NEURAL NETWORKS VIA PROVABLE GRADIENT DESCENT WITH RANDOM INITIALIZATION
Xia, Shuhao
Shi, Yuanming
2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2019, : 5616 - 5620
[33] The Dynamics of Gradient Descent for Overparametrized Neural Networks
Satpathi, Siddhartha
Srikant, R.
LEARNING FOR DYNAMICS AND CONTROL, VOL 144, 2021, 144
[34] Applying Gradient Descent in Convolutional Neural Networks
Cui, Nan
2ND INTERNATIONAL CONFERENCE ON MACHINE VISION AND INFORMATION TECHNOLOGY (CMVIT 2018), 2018, 1004
[35] Deep learning in spiking neural networks
Tavanaei, Amirhossein
Ghodrati, Masoud
Kheradpisheh, Saeed Reza
Masquelier, Timothee
Maida, Anthony
NEURAL NETWORKS, 2019, 111 : 47 - 63
[36] Learning algorithm for spiking neural networks
Amin, HH
Fujii, RH
ADVANCES IN NATURAL COMPUTATION, PT 1, PROCEEDINGS, 2005, 3610 : 456 - 465
[37] Supervised learning with spiking neural networks
Xin, JG
Embrechts, MJ
IJCNN'01: INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-4, PROCEEDINGS, 2001, : 1772 - 1777
[38] Federated Learning With Spiking Neural Networks
Venkatesha, Yeshwanth
Kim, Youngeun
Tassiulas, Leandros
Panda, Priyadarshini
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2021, 69 : 6183 - 6194
[39] Efficient learning in spiking neural networks
Rast, Alexander
Aoun, Mario Antoine
Elia, Eleni G.
Crook, Nigel
NEUROCOMPUTING, 2024, 597
[40] Gradient Learning in Networks of Smoothly Spiking Neurons
Sima, Jiri
ADVANCES IN NEURO-INFORMATION PROCESSING, PT II, 2009, 5507 : 179 - 186

← 1 2 3 4 5 →