The Global Landscape of Neural Networks: An Overview

被引：59

作者：

Sun, Ruoyu ^{[1
,2
,3
,4
]}

Li, Dawei ^{[1
]}

Liang, Shiyu ^{[5
]}

Ding, Tian ^{[6
]}

Srikant, Rayadurgam ^{[2
,3
]}

机构：

[1] Univ Illinois Urbana Champaign UIUC, Dept Ind & Enterprise Syst Engn, Champaign, IL 61820 USA

[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Champaign, IL 61820 USA

[3] Univ Illinois Urbana Champaign UIUC, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[4] Stanford Univ, Stanford, CA 94305 USA

[5] Univ Illinois Urbana Champaign UIUC, Champaign, IL USA

[6] Chinese Univ Hong Kong, Hong Kong, Peoples R China

来源：

IEEE SIGNAL PROCESSING MAGAZINE | 2020年 / 37卷 / 05期

关键词：

Optimization; Signal processing algorithms; Training; Biological neural networks; Convergence; Machine learning;

D O I：

10.1109/MSP.2020.3004124

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

引用

页码：95 / 108

页数：14

共 50 条

[31] Overview of Bifurcation Dynamics in Neural Networks
Xiao, Min
Lu, Yun-Xiang
Yu, Wen-Wu
Zheng, Wei-Xin
Zidonghua Xuebao/Acta Automatica Sinica, 2025, 51 (01): : 72 - 89
[32] Graph Neural Networks: A bibliometrics overview
Keramatfar, Abdalsamad
Rafiee, Mohadeseh
Amirkhani, Hossein
MACHINE LEARNING WITH APPLICATIONS, 2022, 10
[33] An Overview on the Application of Graph Neural Networks in Wireless Networks
He, Shiwen
Xiong, Shaowen
Ou, Yeyu
Zhang, Jian
Wang, Jiaheng
Huang, Yongming
Zhang, Yaoxue
IEEE Open Journal of the Communications Society, 2021, 2 : 2547 - 2565
[34] An Overview on the Application of Graph Neural Networks in Wireless Networks
He, Shiwen
Xiong, Shaowen
Ou, Yeyu
Zhang, Jian
Wang, Jiaheng
Huang, Yongming
Zhang, Yaoxue
IEEE OPEN JOURNAL OF THE COMMUNICATIONS SOCIETY, 2021, 2 : 2547 - 2565
[35] Fuzzy methods and neural networks in landscape modeling
Wieland, R
Lutze, G
Hoffmann, J
6TH MEETING OF THE WG TOOLS FOR SIMULATION AND MODELLING IN ENVIRONMENTAL APPLICATIONS, 1996, 5829 : 49 - 59
[36] A GLOBAL OVERVIEW OF COMPLEX NETWORKS RESEARCH ACTIVITIES
Ma, Fei-cheng
Lyu, Peng-hui
Liu, Xiang
14TH INTERNATIONAL SOCIETY OF SCIENTOMETRICS AND INFORMETRICS CONFERENCE (ISSI), 2013, : 831 - 846
[37] An Overview of Paraconsistent Artificial Neural Networks and Applications
Abe, Jair Minoro
Lopes, Helder F. S.
Nakamatsu, Kazumi
ADVANCES IN KNOWLEDGE-BASED AND INTELLIGENT INFORMATION AND ENGINEERING SYSTEMS, 2012, 243 : 1350 - 1359
[38] On the cover. Neural networks. An overview
Jansson, Peter A.
Analytical Chemistry, 1991, 63 (06)
[39] Spiking Neural Networks for Computational Intelligence: An Overview
Dora, Shirin
Kasabov, Nikola
BIG DATA AND COGNITIVE COMPUTING, 2021, 5 (04)
[40] Convolutional neural networks: an overview and application in radiology
Rikiya Yamashita
Mizuho Nishio
Richard Kinh Gian Do
Kaori Togashi
Insights into Imaging, 2018, 9 : 611 - 629

← 1 2 3 4 5 →