On the overfly algorithm in deep learning of neural networks

被引：2

作者：

Tsygvintsev, Alexei ^{[1
]}

机构：

[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France

来源：

APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷

关键词：

Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;

D O I：

10.1016/j.amc.2018.12.055

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.

引用

页码：348 / 358

页数：11

共 50 条

[41] Learning Structured Sparsity in Deep Neural Networks
Wen, Wei
Wu, Chunpeng
Wang, Yandan
Chen, Yiran
Li, Hai
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016), 2016, 29
[42] Evolving Deep Neural Networks for Continuous Learning
Atamanczuk, Bruna
Karadas, Kurt Arve Skipenes
Agrawal, Bikash
Chakravorty, Antorweep
FRONTIERS OF ARTIFICIAL INTELLIGENCE, ETHICS, AND MULTIDISCIPLINARY APPLICATIONS, FAIEMA 2023, 2024, : 3 - 16
[43] Neural networks and deep learning: a brief introduction
Georgevici, Adrian Iustin
Terblanche, Marius
INTENSIVE CARE MEDICINE, 2019, 45 (05) : 712 - 714
[44] Representational Distance Learning for Deep Neural Networks
McClure, Patrick
Kriegeskorte, Nikolaus
FRONTIERS IN COMPUTATIONAL NEUROSCIENCE, 2016, 10
[45] Learning hidden elasticity with deep neural networks
Chen, Chun-Teh
Gu, Grace X.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2021, 118 (31)
[46] Evolutionary neural networks for deep learning: a review
Ma, Yongjie
Xie, Yirong
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2022, 13 (10) : 3001 - 3018
[47] Deep Learning for Epidemiologists: An Introduction to Neural Networks
Serghiou, Stylianos
Rough, Kathryn
AMERICAN JOURNAL OF EPIDEMIOLOGY, 2023, 192 (11) : 1904 - 1916
[48] Learning Sparse Patterns in Deep Neural Networks
Wen, Weijing
Yang, Fan
Su, Yangfeng
Zhou, Dian
Zeng, Xuan
2019 IEEE 13TH INTERNATIONAL CONFERENCE ON ASIC (ASICON), 2019,
[49] Variational tensor neural networks for deep learning
Jahromi, Saeed S.
Orus, Roman
SCIENTIFIC REPORTS, 2024, 14 (01):
[50] Piecewise linear neural networks and deep learning
Tao, Qinghua
Li, Li
Huang, Xiaolin
Xi, Xiangming
Wang, Shuning
Suykens, Johan A. K.
NATURE REVIEWS METHODS PRIMERS, 2022, 2 (01):

← 1 2 3 4 5 →