On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [11] Deep Learning with Random Neural Networks
    Gelenbe, Erol
    Yin, Yongha
    2016 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2016, : 1633 - 1638
  • [12] Deep Learning with Random Neural Networks
    Gelenbe, Erol
    Yin, Yongha
    PROCEEDINGS OF SAI INTELLIGENT SYSTEMS CONFERENCE (INTELLISYS) 2016, VOL 2, 2018, 16 : 450 - 462
  • [13] Deep learning in spiking neural networks
    Tavanaei, Amirhossein
    Ghodrati, Masoud
    Kheradpisheh, Saeed Reza
    Masquelier, Timothee
    Maida, Anthony
    NEURAL NETWORKS, 2019, 111 : 47 - 63
  • [14] Deep learning in neural networks: An overview
    Schmidhuber, Juergen
    NEURAL NETWORKS, 2015, 61 : 85 - 117
  • [15] Artificial neural networks and deep learning
    Geubbelmans, Melvin
    Rousseau, Axel-Jan
    Burzykowski, Tomasz
    Valkenborg, Dirk
    AMERICAN JOURNAL OF ORTHODONTICS AND DENTOFACIAL ORTHOPEDICS, 2024, 165 (02) : 248 - 251
  • [16] Shortcut learning in deep neural networks
    Robert Geirhos
    Jörn-Henrik Jacobsen
    Claudio Michaelis
    Richard Zemel
    Wieland Brendel
    Matthias Bethge
    Felix A. Wichmann
    Nature Machine Intelligence, 2020, 2 : 665 - 673
  • [17] Fast learning in Deep Neural Networks
    Chandra, B.
    Sharma, Rajesh K.
    NEUROCOMPUTING, 2016, 171 : 1205 - 1215
  • [18] Deep associative learning for neural networks
    Liu, Jia
    Zhang, Wenhua
    Liu, Fang
    Xiao, Liang
    NEUROCOMPUTING, 2021, 443 (443) : 222 - 234
  • [19] Collaborative Learning for Deep Neural Networks
    Song, Guocong
    Chai, Wei
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), 2018, 31
  • [20] Big learning and deep neural networks
    Montavon, Grégoire
    Müller, Klaus-Robert
    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2012, 7700 LECTURE NO : 419 - 420
12345