The Global Landscape of Neural Networks: An Overview

被引：59

作者：

Sun, Ruoyu ^{[1
,2
,3
,4
]}

Li, Dawei ^{[1
]}

Liang, Shiyu ^{[5
]}

Ding, Tian ^{[6
]}

Srikant, Rayadurgam ^{[2
,3
]}

机构：

[1] Univ Illinois Urbana Champaign UIUC, Dept Ind & Enterprise Syst Engn, Champaign, IL 61820 USA

[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Champaign, IL 61820 USA

[3] Univ Illinois Urbana Champaign UIUC, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[4] Stanford Univ, Stanford, CA 94305 USA

[5] Univ Illinois Urbana Champaign UIUC, Champaign, IL USA

[6] Chinese Univ Hong Kong, Hong Kong, Peoples R China

来源：

IEEE SIGNAL PROCESSING MAGAZINE | 2020年 / 37卷 / 05期

关键词：

Optimization; Signal processing algorithms; Training; Biological neural networks; Convergence; Machine learning;

D O I：

10.1109/MSP.2020.3004124

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

引用

页码：95 / 108

页数：14

共 50 条

[41] NEURAL NETWORKS AND OPERATIONS-RESEARCH - AN OVERVIEW
BURKE, LI
IGNIZIO, JP
COMPUTERS & OPERATIONS RESEARCH, 1992, 19 (3-4) : 179 - 189
[42] AN OVERVIEW OF ARTIFICIAL NEURAL NETWORKS APPLICATION IN TRANSPORTATION
Zenina, Nadezda
Merkuryev, Yuri
MENDEL 2008, 2008, : 12 - 16
[43] A brief overview and introduction to artificial neural networks
Buscema, M
SUBSTANCE USE & MISUSE, 2002, 37 (8-10) : 1093 - 1148
[44] Overview of RFID Applications Utilizing Neural Networks
Durtschi, Barrett D.
Chrysler, Andrew M.
IEEE JOURNAL OF RADIO FREQUENCY IDENTIFICATION, 2024, 8 : 801 - 810
[45] Overview of Visualization Methods for Artificial Neural Networks
Matveev, S. A.
Oseledets, I., V
Ponomarev, E. S.
Chertkov, A., V
COMPUTATIONAL MATHEMATICS AND MATHEMATICAL PHYSICS, 2021, 61 (05) : 887 - 899
[46] Overview of the Research Status on Artificial Neural Networks
Wang Xin-gang
PROCEEDINGS OF THE 2ND INTERNATIONAL FORUM ON MANAGEMENT, EDUCATION AND INFORMATION TECHNOLOGY APPLICATION (IFMEITA 2017), 2017, 130 : 351 - 356
[47] Hardware Compilation of Deep Neural Networks: An Overview
Zhao, Ruizhe
Liu, Shuanglong
Ng, Ho-Cheung
Wang, Erwei
Davis, James J.
Niu, Xinyu
Wang, Xiwei
Shi, Huifeng
Constantinides, George A.
Cheung, Peter Y. K.
Luk, Wayne
2018 IEEE 29TH INTERNATIONAL CONFERENCE ON APPLICATION-SPECIFIC SYSTEMS, ARCHITECTURES AND PROCESSORS (ASAP), 2018, : 120 - 127
[48] Deep Neural Networks in Machine Translation: An Overview
Zhang, Jiajun
Zong, Chengqing
IEEE INTELLIGENT SYSTEMS, 2015, 30 (05) : 16 - 25
[49] Overview of the applications of neural networks in process engineering
Mirzai, A.R.
Leigh, J.R.
Computing and Control Engineering Journal, 1992, 3 (03): : 105 - 108
[50] Overview of Visualization Methods for Artificial Neural Networks
S. A. Matveev
I. V. Oseledets
E. S. Ponomarev
A. V. Chertkov
Computational Mathematics and Mathematical Physics, 2021, 61 : 887 - 899

← 1 2 3 4 5 →