The Global Landscape of Neural Networks: An Overview

被引：59

作者：

Sun, Ruoyu ^{[1
,2
,3
,4
]}

Li, Dawei ^{[1
]}

Liang, Shiyu ^{[5
]}

Ding, Tian ^{[6
]}

Srikant, Rayadurgam ^{[2
,3
]}

机构：

[1] Univ Illinois Urbana Champaign UIUC, Dept Ind & Enterprise Syst Engn, Champaign, IL 61820 USA

[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Champaign, IL 61820 USA

[3] Univ Illinois Urbana Champaign UIUC, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[4] Stanford Univ, Stanford, CA 94305 USA

[5] Univ Illinois Urbana Champaign UIUC, Champaign, IL USA

[6] Chinese Univ Hong Kong, Hong Kong, Peoples R China

来源：

IEEE SIGNAL PROCESSING MAGAZINE | 2020年 / 37卷 / 05期

关键词：

Optimization; Signal processing algorithms; Training; Biological neural networks; Convergence; Machine learning;

D O I：

10.1109/MSP.2020.3004124

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

引用

页码：95 / 108

页数：14

共 50 条

[21] Nonequilibrium landscape theory of neural networks
Yan, Han
Zhao, Lei
Hu, Liang
Wang, Xidi
Wang, Erkang
Wang, Jin
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2013, 110 (45) : E4185 - E4194
[22] Landscape Classification with Deep Neural Networks
Buscombe, Daniel
Ritchie, Andrew C.
GEOSCIENCES, 2018, 8 (07)
[23] Global overview of patenting landscape in unmanned aerial vehicles
Philippe, Gorry
Kotsemir, Maxim
17TH INTERNATIONAL CONFERENCE ON SCIENTOMETRICS & INFORMETRICS (ISSI2019), VOL II, 2019, : 2682 - 2683
[24] An Overview of Neural Networks in Simulation Soccer
Plant, William
Schaefer, Gerald
2009 WORLD CONGRESS ON NATURE & BIOLOGICALLY INSPIRED COMPUTING (NABIC 2009), 2009, : 1619 - +
[25] Deep learning in neural networks: An overview
Schmidhuber, Juergen
NEURAL NETWORKS, 2015, 61 : 85 - 117
[26] Overview of hybrid optical neural networks
Jutamulia, S
Fu, FTS
OPTICS AND LASER TECHNOLOGY, 1996, 28 (02): : 59 - 72
[27] An overview of research in certification of neural networks
Morgan, G
Austin, J
NEURAL NETWORKS - PRODUCING DEPENDABLE SYSTEMS, CONFERENCE PROCEEDINGS, 1996, 95 (973): : 3 - 9
[28] An overview on supervised neural networks for structures
Sperduti, A
1997 IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS, VOLS 1-4, 1997, : 2550 - 2554
[29] NEURAL NETWORKS - CONFERENCE UPDATE AND OVERVIEW
ELIOT, LB
IEEE EXPERT-INTELLIGENT SYSTEMS & THEIR APPLICATIONS, 1987, 2 (04): : 12 - 14
[30] Recurrent neural networks: Overview and perspectives
Michel, AN
PROCEEDINGS OF THE 2003 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOL III: GENERAL & NONLINEAR CIRCUITS AND SYSTEMS, 2003, : 474 - 477

← 1 2 3 4 5 →