The Global Landscape of Neural Networks: An Overview

被引：59

作者：

Sun, Ruoyu ^{[1
,2
,3
,4
]}

Li, Dawei ^{[1
]}

Liang, Shiyu ^{[5
]}

Ding, Tian ^{[6
]}

Srikant, Rayadurgam ^{[2
,3
]}

机构：

[1] Univ Illinois Urbana Champaign UIUC, Dept Ind & Enterprise Syst Engn, Champaign, IL 61820 USA

[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Champaign, IL 61820 USA

[3] Univ Illinois Urbana Champaign UIUC, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[4] Stanford Univ, Stanford, CA 94305 USA

[5] Univ Illinois Urbana Champaign UIUC, Champaign, IL USA

[6] Chinese Univ Hong Kong, Hong Kong, Peoples R China

来源：

IEEE SIGNAL PROCESSING MAGAZINE | 2020年 / 37卷 / 05期

关键词：

Optimization; Signal processing algorithms; Training; Biological neural networks; Convergence; Machine learning;

D O I：

10.1109/MSP.2020.3004124

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

引用

页码：95 / 108

页数：14

共 50 条

[1] Overview of global biofuels landscape
不详
INTERNATIONAL SUGAR JOURNAL, 2015, 117 (1402): : 710 - +
[2] Overview of global biofuels landscape
Int. Sugar J., 1402
[3] NEURAL NETWORKS - AN OVERVIEW
JANSSON, PA
ANALYTICAL CHEMISTRY, 1991, 63 (06) : A357 - A362
[4] AN OVERVIEW OF NEURAL NETWORKS
GREENWOOD, D
BEHAVIORAL SCIENCE, 1991, 36 (01): : 1 - 33
[5] Application of artificial neural networks in global climate change and ecological research:An overview
LIU ZeLin1
2 Institute of Environment Sciences
ChineseScienceBulletin, 2010, 55 (34) : 3853 - 3863
[6] Application of artificial neural networks in global climate change and ecological research: An overview
Liu ZeLin
Peng ChangHui
Xiang WenHua
Tian DaLun
Deng XiangWen
Zhao MeiFang
CHINESE SCIENCE BULLETIN, 2010, 55 (34): : 3853 - 3863
[7] Overview of recurrent neural networks
Liu J.-W.
Song Z.-Y.
Kongzhi yu Juece/Control and Decision, 2022, 37 (11): : 2753 - 2768
[8] An overview on rough neural networks
Hongmei Liao
Shifei Ding
Miaomiao Wang
Gang Ma
Neural Computing and Applications, 2016, 27 : 1805 - 1816
[9] ARTIFICIAL NEURAL NETWORKS - AN OVERVIEW
PARKER, DB
ANDREW SEYBOLDS OUTLOOK ON PROFESSIONAL COMPUTING, 1988, 6 (10): : 23 - 25
[10] Enhanced neural networks:: An overview
Mingo, LF
Díaz, MA
Gisbert, F
Carrillo, JD
KNOWLEDGE-BASED INTELLIGENT INFORMATION ENGINEERING SYSTEMS & ALLIED TECHNOLOGIES, PTS 1 AND 2, 2001, 69 : 1170 - 1173

← 1 2 3 4 5 →