Analytical form of Fisher information matrix of bipoloar-activation-function-based multilayer perceptrons

被引：0

作者：

Guo, Weili ^{[1
,2
,3
]}

Xie, Liping ^{[3
]}

Fu, Zhenyong ^{[1
,2
]}

Guo, Jianhui ^{[1
,2
]}

Pang, Guochen ^{[4
]}

Yang, Jian ^{[1
,2
]}

机构：

[1] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Minist Educ, PCA Lab,Key Lab Intelligent Percept & Syst High D, Nanjing, Peoples R China

[2] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Jiangsu Key Lab Image & Video Understanding Socia, Nanjing, Peoples R China

[3] Southeast Univ, Sch Automat, Key Lab Measurement & Control CSE, Minist Educ, Nanjing, Peoples R China

[4] Linyi Univ, Sch Automat & Elect Engn, Linyi, Shandong, Peoples R China

来源：

2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) | 2020年

关键词：

Fisher information matrix; multilayer perceptrons; singularity; bipolar error function; feedforward neural networks; GRADIENT LEARNING ALGORITHMS; NATURAL GRADIENT; DYNAMICS; SINGULARITIES;

D O I：

10.1109/ijcnn48605.2020.9207059

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

For the widely used multilayer perceptrons (MLPs), the existed singularities in the parameter space have seriously affected the learning dynamics of MLPs, which cause several singular learning behaviors. Since the Fisher information matrix (FIM) plays a significant role in analyzing the singular learning dyanmics of MLPs, it is very worthy to obtain the analytical form of FIM to do further investigation. In this paper, by choosing the bipolar error function as the activation function, the analytical form of FIM are obtained, where the validity of the obtained results are verified by taking three experiments.

引用

页数：8

共 50 条

[21] Optimal Design of Moving Short Base-lines Based on Fisher Information Matrix
Shang, Zhigang
Ma, Xiaochuan
Li, Tieli
Lin, Jincheng
2014 FOURTH INTERNATIONAL CONFERENCE ON INSTRUMENTATION AND MEASUREMENT, COMPUTER, COMMUNICATION AND CONTROL (IMCCC), 2014, : 16 - 20
[22] Model-based geometry optimization of a Nitsch cell using the Fisher information matrix
Bertakis, Evangelos
Kalem, Murat
Pfennig, Andreas
CHEMICAL ENGINEERING SCIENCE, 2008, 63 (19) : 4881 - 4887
[23] Fractal analytical approach of urban form based on spatial correlation function
Chen, Yanguang
CHAOS SOLITONS & FRACTALS, 2013, 49 : 47 - 60
[24] Information Matrix-Based Adaptive Sampling in Hull Form Optimisation
Ouyang, Xuyu
Chang, Haichao
Feng, Baiwei
Liu, Zuyuan
Zhan, Chengsheng
Cheng, Xide
JOURNAL OF MARINE SCIENCE AND ENGINEERING, 2021, 9 (09)
[25] Fisher-Information-Matrix-Based USBL Cooperative Location in USV-AUV Networks
Wang, Ziyuan
Xu, Jingzehua
Feng, Yuanzhe
Wang, Yijing
Xie, Guanwen
Hou, Xiangwang
Men, Wei
Ren, Yong
SENSORS, 2023, 23 (17)
[26] Optimal design of experiments for optimization-based model calibration using Fisher information matrix
Jung, Yongsu
Lee, Ikjin
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2021, 216
[27] Sparsified federated learning with differential privacy for intrusion detection in VANETs based on Fisher Information Matrix
Chen, Rui
Chen, Xiaoyu
Zhao, Jing
PLOS ONE, 2024, 19 (04):
[28] Characteristic Function-Based Lower Bounds for Fisher Information Under Arbitrary Parametrization
Dulek, Berkan
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS, 2017, 53 (01) : 501 - 512
[29] A Multi-Objective Strategy based on Frontier-based Approach and Fisher Information Matrix for Autonomous Exploration
Fang, Zheng
Zhang, Lei
2013 IEEE 3RD ANNUAL INTERNATIONAL CONFERENCE ON CYBER TECHNOLOGY IN AUTOMATION, CONTROL AND INTELLIGENT SYSTEMS (CYBER), 2013, : 96 - 101
[30] The finite state projection based Fisher information matrix approach to estimate information and optimize single-cell experiments
Fox, Zachary R.
Munsky, Brian
PLOS COMPUTATIONAL BIOLOGY, 2019, 15 (01)

← 1 2 3 4 5 →