Reinforcement Learning in Memristive Spiking Neural Networks through Modulation of ReSuMe

被引：2

作者：

Ji, Xun ^{[1
]}

Zhang, Yaozhong ^{[1
]}

Li, Chuxi ^{[1
]}

Wu, Tanghong ^{[1
]}

Hu, Xiaofang ^{[1
]}

机构：

[1] Southwest Univ, Coll Comp & Informat Sci, Chongqing 400715, Peoples R China

来源：

ADVANCES IN MATERIALS, MACHINERY, ELECTRONICS III | 2019年 / 2073卷

基金：

中国博士后科学基金; 中国国家自然科学基金;

关键词：

Reinforcement Learning; Spiking Neural Network; Remote Supervised Method; Memristor;

D O I：

10.1063/1.5090748

中图分类号：

T [工业技术];

学科分类号：

08 ;

摘要：

In this paper, a novel hardware-friendly reinforcement learning algorithm based on memristive spiking neural networks (MSNN-RL) is proposed. Neurons for spike coding are designed specifically to complete transformation between analog data and discrete spikes. Then, remote supervised method (ReSuMe) is used to combine SNN with basic reforcement learing (Sarsa). Besides, bionic memristive snynapses are designed to speed up ReSuMe. Furthermore, the circuit scheme of MSNN-RL is designed with modulation of memristor synapses. Finally, the application of MSNN-RL in acrobot system is discussed. Simulation results and analysis verify the effectiveness of the proposed algorithm (MSNN-RL) and show it is superior to traditional apporach.

引用

页数：9

共 50 条

[1] Analysis of the ReSuMe learning process for spiking neural networks
Ponulak, Filip
INTERNATIONAL JOURNAL OF APPLIED MATHEMATICS AND COMPUTER SCIENCE, 2008, 18 (02) : 117 - 127
[2] Backpropagating Errors Through Memristive Spiking Neural Networks
Zhou, Peng
Choi, Dong-Uk
Kang, Sung-Mo
Eshraghian, Jason K.
2023 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, ISCAS, 2023,
[3] A compound memristive synapse model for statistical learning through STDP in spiking neural networks
Bill, Johannes
Legenstein, Robert
FRONTIERS IN NEUROSCIENCE, 2014, 8
[4] Learning in neural networks by reinforcement of irregular spiking
Xie, XH
Seung, HS
PHYSICAL REVIEW E, 2004, 69 (04): : 10
[5] A reinforcement learning algorithm for spiking neural networks
Florian, RV
Seventh International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Proceedings, 2005, : 299 - 306
[6] Digital Hardware Implementation of ReSuMe Learning Algorithm for Spiking Neural Networks
Khatiboun, Dario Fernandez
Rezaeiyan, Yasser
Ronchini, Margherita
Sadeghi, Maryam
Zamani, Milad
Moradi, Farshad
2023 45TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE & BIOLOGY SOCIETY, EMBC, 2023,
[7] Effects of memristive synapse radiation interactions on learning in spiking neural networks
Sumedha Gandharava Dahl
Robert C. Ivans
Kurtis D. Cantley
SN Applied Sciences, 2021, 3
[8] Effects of memristive synapse radiation interactions on learning in spiking neural networks
Dahl, Sumedha Gandharava
Ivans, Robert C.
Cantley, Kurtis D.
SN APPLIED SCIENCES, 2021, 3 (05):
[9] Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting
Ponulak, Filip
Kasinski, Andrzej
NEURAL COMPUTATION, 2010, 22 (02) : 467 - 510
[10] A Novel Reinforcement Learning Algorithm Based on Multilayer Memristive Spiking Neural Network With Applications
Zhang Y.-Z.
Hu X.-F.
Zhou Y.
Duan S.-K.
Zidonghua Xuebao/Acta Automatica Sinica, 2019, 45 (08): : 1536 - 1547

← 1 2 3 4 5 →