Analytical form of Fisher information matrix of bipoloar-activation-function-based multilayer perceptrons

被引：0

作者：

Guo, Weili ^{[1
,2
,3
]}

Xie, Liping ^{[3
]}

Fu, Zhenyong ^{[1
,2
]}

Guo, Jianhui ^{[1
,2
]}

Pang, Guochen ^{[4
]}

Yang, Jian ^{[1
,2
]}

机构：

[1] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Minist Educ, PCA Lab,Key Lab Intelligent Percept & Syst High D, Nanjing, Peoples R China

[2] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Jiangsu Key Lab Image & Video Understanding Socia, Nanjing, Peoples R China

[3] Southeast Univ, Sch Automat, Key Lab Measurement & Control CSE, Minist Educ, Nanjing, Peoples R China

[4] Linyi Univ, Sch Automat & Elect Engn, Linyi, Shandong, Peoples R China

来源：

2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | 2020年

关键词：

Fisher information matrix; multilayer perceptrons; singularity; bipolar error function; feedforward neural networks; GRADIENT LEARNING ALGORITHMS; NATURAL GRADIENT; DYNAMICS; SINGULARITIES;

D O I：

10.1109/ijcnn48605.2020.9207059

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

For the widely used multilayer perceptrons (MLPs), the existed singularities in the parameter space have seriously affected the learning dynamics of MLPs, which cause several singular learning behaviors. Since the Fisher information matrix (FIM) plays a significant role in analyzing the singular learning dyanmics of MLPs, it is very worthy to obtain the analytical form of FIM to do further investigation. In this paper, by choosing the bipolar error function as the activation function, the analytical form of FIM are obtained, where the validity of the obtained results are verified by taking three experiments.

引用

页数：8

共 50 条

[41] Mining subsidence monitoring using distributed scatterers InSAR based on Goldstein filter and Fisher information matrix-weighted optimization
Tian, Zeming
Zhao, Lifeng
Fan, Hongdong
Lin, Tao
Li, Tao
NATURAL HAZARDS, 2024, 120 (05) : 4205 - 4231
[42] UAV Path Optimization for Angle-Only Self-Localization and Target Tracking Based on the Bayesian Fisher Information Matrix
Dogancay, Kutluyil
Hmam, Hatem
SENSORS, 2024, 24 (10)
[43] Mining subsidence monitoring using distributed scatterers InSAR based on Goldstein filter and Fisher information matrix-weighted optimization
Zeming Tian
Lifeng Zhao
Hongdong Fan
Tao Lin
Tao Li
Natural Hazards, 2024, 120 : 4205 - 4231
[44] TESTING THE FORM OF A DECISION-MAKERS MULTIATTRIBUTE VALUE FUNCTION BASED ON PAIRWISE PREFERENCE INFORMATION
SALMINEN, P
KORHONEN, P
WALLENIUS, J
JOURNAL OF THE OPERATIONAL RESEARCH SOCIETY, 1989, 40 (03) : 299 - 302
[45] Parameter estimation of inspiralling compact binaries in ground-based detectors: comparison between Monte Carlo simulations and the Fisher information matrix
Cokelaer, T.
CLASSICAL AND QUANTUM GRAVITY, 2008, 25 (18)
[46] Accurate calculation and Matlab based fast realization of merit function's Hesse matrix for the design of multilayer optical coating
吴素勇
龙兴武
杨开勇
Optoelectronics Letters, 2009, (05) : 359 - 363
[47] Accurate calculation and Matlab based fast realization of merit function's Hesse matrix for the design of multilayer optical coating
Wu S.-Y.
Long X.-W.
Yang K.-Y.
Optoelectronics Letters, 2009, 5 (5) : 359 - 363
[48] Gradient-based Neural Network for Online Solution of Lyapunov Matrix Equation with Li Activation Function
Wang, Shiheng
Dai, Shidong
Wang, Ke
PROCEEDINGS OF THE 4TH INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY AND MANAGEMENT INNOVATION, 2015, 28 : 955 - 959
[49] Analytical solutions to the matrix inequalities in the robust control scheme based on implicit Lyapunov function for spacecraft rendezvous on elliptical orbit
Tian, Xiwen
Jia, Yingmin
IET CONTROL THEORY AND APPLICATIONS, 2017, 11 (12): : 1983 - 1991
[50] A new artificial lateral line attitude perception method based on mixed activation function-multilayer perceptron (MAF-MLP)
Li, Gongbo
Liu, Guijie
Jin, Bei
Wang, Wenqian
Fang, Xin
OCEAN ENGINEERING, 2023, 288

← 1 2 3 4 5 →