The Global Landscape of Neural Networks: An Overview

被引：59

作者：

Sun, Ruoyu ^{[1
,2
,3
,4
]}

Li, Dawei ^{[1
]}

Liang, Shiyu ^{[5
]}

Ding, Tian ^{[6
]}

Srikant, Rayadurgam ^{[2
,3
]}

机构：

[1] Univ Illinois Urbana Champaign UIUC, Dept Ind & Enterprise Syst Engn, Champaign, IL 61820 USA

[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Champaign, IL 61820 USA

[3] Univ Illinois Urbana Champaign UIUC, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[4] Stanford Univ, Stanford, CA 94305 USA

[5] Univ Illinois Urbana Champaign UIUC, Champaign, IL USA

[6] Chinese Univ Hong Kong, Hong Kong, Peoples R China

来源：

IEEE SIGNAL PROCESSING MAGAZINE | 2020年 / 37卷 / 05期

关键词：

Optimization; Signal processing algorithms; Training; Biological neural networks; Convergence; Machine learning;

D O I：

10.1109/MSP.2020.3004124

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

引用

页码：95 / 108

页数：14

共 50 条

[11] Randomness in neural networks: an overview
Scardapane, Simone
Wang, Dianhui
WILEY INTERDISCIPLINARY REVIEWS-DATA MINING AND KNOWLEDGE DISCOVERY, 2017, 7 (02)
[12] Artificial Neural Networks: An Overview
Micheli-Tzanakou, Evangelia
NETWORK-COMPUTATION IN NEURAL SYSTEMS, 2011, 22 (1-4) : 208 - 230
[13] A practical overview of neural networks
Burke, L
Ignizio, JP
JOURNAL OF INTELLIGENT MANUFACTURING, 1997, 8 (03) : 157 - 165
[14] Overview of Capsule Neural Networks
Sun, Zengguo
Zhao, Guodong
Scherer, Rafal
Wei, Wei
Wozniak, Marcin
JOURNAL OF INTERNET TECHNOLOGY, 2022, 23 (01): : 33 - 44
[15] NEURAL NETWORKS - A BIASED OVERVIEW
DOMANY, E
JOURNAL OF STATISTICAL PHYSICS, 1988, 51 (5-6) : 743 - 775
[16] An overview on rough neural networks
Liao, Hongmei
Ding, Shifei
Wang, Miaomiao
Ma, Gang
NEURAL COMPUTING & APPLICATIONS, 2016, 27 (07): : 1805 - 1816
[17] Dynamic neural networks: An overview
Sinha, NK
Gupta, MM
Rao, DH
PROCEEDINGS OF IEEE INTERNATIONAL CONFERENCE ON INDUSTRIAL TECHNOLOGY 2000, VOLS 1 AND 2, 2000, : 491 - 496
[18] A practical overview of neural networks
LAURA BURKE
JAMES P. IGNIZIO
Journal of Intelligent Manufacturing, 1997, 8 : 157 - 165
[19] Genetic landscape of gallbladder cancer: Global overview
Mehrotra, Ravi
Tulsyan, Sonam
Hussain, Showket
Mittal, Balraj
Saluja, Sundeep Singh
Singh, Sandeep
Tanwar, Pranay
Khan, Asiya
Javle, Milind
Hassan, Manal M.
Pant, Shubham
De Aretxabala, Xabier
Sirohi, Bhawna
Rajaraman, Preetha
Kaur, Tanvir
Rath, G. K.
MUTATION RESEARCH-REVIEWS IN MUTATION RESEARCH, 2018, 778 : 61 - 71
[20] An Overview of the Security Landscape of Virtual Mobile Networks
Ahmad, Ijaz
Pinola, Jarno
Harjula, Ilkka
Suomalainen, Jani
Harjula, Erkki
Huusko, Jyrki
Kumar, Tanesh
IEEE ACCESS, 2021, 9 : 169014 - 169030

← 1 2 3 4 5 →