Nonlinear Bayesian Filters for Training Recurrent Neural Networks

被引：0

作者：

Arasaratnam, Ienkaran ^{[1
]}

Haykin, Simon ^{[1
]}

机构：

[1] McMaster Univ, Dept Elect & Comp Engn, Cognit Syst Lab, Hamilton, ON L8S 4K1, Canada

来源：

MICAI 2008: ADVANCES IN ARTIFICIAL INTELLIGENCE, PROCEEDINGS | 2008年 / 5317卷

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In this paper; we present nonlinear Bayesian filters for training recurrent neural networks with a special emphasis oil a novel, more accurate, derivative-free member of the approximate Bayesian filter family called the cubature Kalman filter. We discuss the theory of Bayesian filters, which is rooted in the state-space modeling of the dynamic system in question and the linear estimation principle. For improved numerical stability and optimal performance during training period; a number of techniques of how to tune Bayesian filters is suggested. We compare the predictability of various Bayesian filter-trained recurrent neural networks using a chaotic time-series. From the empirical results; we conclude that the performance may be greatly improved by the new square-root cubature Kalman filter.

引用

页码：12 / 33

页数：22

共 50 条

[1] Recurrent Neural Networks Training Using Derivative Free Nonlinear Bayesian Filters
Todorovic, Branimir
Stankovic, Miomir
Moraga, Claudio
COMPUTATIONAL INTELLIGENCE, IJCCI 2014, 2016, 620 : 383 - 410
[2] Gaussian sum filters for recurrent neural networks training
Todorovic, Branimir
Stankovic, Miomir
Moraga, Claudio
NEUREL 2006: EIGHT SEMINAR ON NEURAL NETWORK APPLICATIONS IN ELECTRICAL ENGINEERING, PROCEEDINGS, 2006, : 53 - +
[3] Parallel nonlinear adaptive digital filters using recurrent neural networks
Cao, JT
Yahagi, T
ELECTRONICS AND COMMUNICATIONS IN JAPAN PART III-FUNDAMENTAL ELECTRONIC SCIENCE, 1997, 80 (12): : 91 - 101
[4] Parallel nonlinear adaptive digital filters using recurrent neural networks
Cao, JT
Yahagi, T
ELECTRONICS AND COMMUNICATIONS IN JAPAN PART III-FUNDAMENTAL ELECTRONIC SCIENCE, 1997, 80 (03): : 83 - 93
[5] Recursive Bayesian Levenberg-Marquardt training of recurrent neural networks
Mirikitani, Derrick
Nikolaev, Nikolay
2007 IEEE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-6, 2007, : 282 - 287
[6] Convolutional Neural Networks with Recurrent Neural Filters
Yang, Yi
2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018), 2018, : 912 - 917
[7] Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks
Sebastian Bitzer
Stefan J. Kiebel
Biological Cybernetics, 2012, 106 : 201 - 217
[8] Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks
Bitzer, Sebastian
Kiebel, Stefan J.
BIOLOGICAL CYBERNETICS, 2012, 106 (4-5) : 201 - 217
[9] On-line learning in recurrent neural networks using nonlinear Kalman filters
Todorovic, B
Stankovic, M
Moraga, C
PROCEEDINGS OF THE 3RD IEEE INTERNATIONAL SYMPOSIUM ON SIGNAL PROCESSING AND INFORMATION TECHNOLOGY, 2003, : 802 - 805
[10] Sparse Bayesian Recurrent Neural Networks
Chatzis, Sotirios P.
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2015, PT II, 2015, 9285 : 359 - 372

← 1 2 3 4 5 →