Analytical form of Fisher information matrix of bipoloar-activation-function-based multilayer perceptrons

被引：0
|
作者：
Guo, Weili [1 ,2 ,3 ]
Xie, Liping [3 ]
Fu, Zhenyong [1 ,2 ]
Guo, Jianhui [1 ,2 ]
Pang, Guochen [4 ]
Yang, Jian [1 ,2 ]
机构：
[1] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Minist Educ, PCA Lab,Key Lab Intelligent Percept & Syst High D, Nanjing, Peoples R China
[2] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Jiangsu Key Lab Image & Video Understanding Socia, Nanjing, Peoples R China
[3] Southeast Univ, Sch Automat, Key Lab Measurement & Control CSE, Minist Educ, Nanjing, Peoples R China
[4] Linyi Univ, Sch Automat & Elect Engn, Linyi, Shandong, Peoples R China
来源：
2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | 2020年
关键词：
Fisher information matrix; multilayer perceptrons; singularity; bipolar error function; feedforward neural networks; GRADIENT LEARNING ALGORITHMS; NATURAL GRADIENT; DYNAMICS; SINGULARITIES;
D O I：
10.1109/ijcnn48605.2020.9207059
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
For the widely used multilayer perceptrons (MLPs), the existed singularities in the parameter space have seriously affected the learning dynamics of MLPs, which cause several singular learning behaviors. Since the Fisher information matrix (FIM) plays a significant role in analyzing the singular learning dyanmics of MLPs, it is very worthy to obtain the analytical form of FIM to do further investigation. In this paper, by choosing the bipolar error function as the activation function, the analytical form of FIM are obtained, where the validity of the obtained results are verified by taking three experiments.
引用
收藏
页数：8
相关论文
共 50 条
  • [1] Fisher Information Matrix of Unipolar Activation Function-Based Multilayer Perceptrons
    Guo, Weili
    Ong, Yew-Soon
    Zhou, Yingjiang
    Hervas, Jaime Rubio
    Song, Aiguo
    Wei, Haikun
    IEEE TRANSACTIONS ON CYBERNETICS, 2019, 49 (08) : 3088 - 3098
  • [2] Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input
    Guo, Weili
    Li, Guangyu
    Lu, Jianfeng
    IEEE ACCESS, 2022, 10 : 128250 - 128262
  • [3] Multilayer perceptrons as function approximators for analytical solutions of the diffusion equation
    Laura D. Campisi
    Computational Geosciences, 2015, 19 : 769 - 780
  • [4] Multilayer perceptrons as function approximators for analytical solutions of the diffusion equation
    Campisi, Laura D.
    COMPUTATIONAL GEOSCIENCES, 2015, 19 (04) : 769 - 780
  • [5] Training multilayer perceptrons via interpolation by superpositions of an activation function
    Li, X
    METMBS'01: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON MATHEMATICS AND ENGINEERING TECHNIQUES IN MEDICINE AND BIOLOGICAL SCIENCES, 2001, : 19 - 24
  • [6] Efficient pruning of multilayer perceptrons using a fuzzy sigmoid activation function
    Soria-Olivas, E
    Martín-Guerrero, JD
    Serrano-López, AJ
    Calpe-Maravilla, J
    Vila-Francés, J
    Camps-Valls, G
    NEUROCOMPUTING, 2006, 69 (7-9) : 909 - 912
  • [7] Interpreting Multilayer Perceptrons Using 3-Valued Activation Function
    Furukawa, Tomoya
    Zhao, Qiangfu
    2017 3RD IEEE INTERNATIONAL CONFERENCE ON CYBERNETICS (CYBCONF), 2017, : 326 - 331
  • [8] Averaged learning equations of error-function-based multilayer perceptrons
    Guo, Weili
    Wei, Haikun
    Zhao, Junsheng
    Zhang, Kanjian
    NEURAL COMPUTING & APPLICATIONS, 2014, 25 (3-4): : 825 - 832
  • [9] Averaged learning equations of error-function-based multilayer perceptrons
    Weili Guo
    Haikun Wei
    Junsheng Zhao
    Kanjian Zhang
    Neural Computing and Applications, 2014, 25 : 825 - 832
  • [10] On a measure of dependence based on Fisher's information matrix
    Zografos, K
    COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 1998, 27 (07) : 1715 - 1728
12345