On the overfly algorithm in deep learning of neural networks

被引：2
|
作者：
Tsygvintsev, Alexei [1 ]
机构：
[1] Ecole Normale Super Lyon, UMPA, 46 Allee Italie, F-69364 Lyon 07, France
来源：
APPLIED MATHEMATICS AND COMPUTATION | 2019年 / 349卷
关键词：
Deep learning; Neural networks; Dynamical systems; Gradient descent; LOCAL MINIMA;
D O I：
10.1016/j.amc.2018.12.055
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
In this paper, we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation. (C) 2019 Elsevier Inc. All rights reserved.
引用
收藏
页码：348 / 358
页数：11
相关论文
共 50 条
  • [1] Supervised Learning Based Algorithm Selection for Deep Neural Networks
    Shi, Shaohuai
    Xu, Pengfei
    Chu, Xiamen
    2017 IEEE 23RD INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS (ICPADS), 2017, : 344 - 351
  • [2] An Unsupervised Learning Algorithm for Deep Recurrent Spiking Neural Networks
    Du, Pangao
    Lin, Xianghong
    Pi, Xiaomei
    Wang, Xiangwen
    2020 11TH IEEE ANNUAL UBIQUITOUS COMPUTING, ELECTRONICS & MOBILE COMMUNICATION CONFERENCE (UEMCON), 2020, : 603 - 607
  • [3] Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks
    Alexandrov I.A.
    Kirichek A.V.
    Kuklin V.Z.
    Chervyakov L.M.
    HighTech and Innovation Journal, 2023, 4 (01): : 157 - 173
  • [4] ABOUT AN ALGORITHM FOR CONSISTENT WEIGHTS INITIALIZATION OF DEEP NEURAL NETWORKS AND NEURAL NETWORKS ENSEMBLE LEARNING
    Drokin, I. S.
    VESTNIK SANKT-PETERBURGSKOGO UNIVERSITETA SERIYA 10 PRIKLADNAYA MATEMATIKA INFORMATIKA PROTSESSY UPRAVLENIYA, 2016, 12 (04): : 66 - 74
  • [5] A Boosting-based Deep Neural Networks Algorithm for Reinforcement Learning
    Wang, Yu
    Jin, Hongxia
    2018 ANNUAL AMERICAN CONTROL CONFERENCE (ACC), 2018, : 1065 - 1071
  • [6] An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks
    Zhu, Xiaolei
    Zhao, Baixin
    Ma, De
    Tang, Huajin
    IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, 2022, 14 (03) : 847 - 856
  • [7] Solving flows of dynamical systems by deep neural networks and a novel deep learning algorithm
    Liao, Guangyuan
    Zhang, Limin
    MATHEMATICS AND COMPUTERS IN SIMULATION, 2022, 202 : 331 - 342
  • [8] Online Deep Learning: Learning Deep Neural Networks on the Fly
    Sahoo, Doyen
    Pham, Quang
    Lu, Jing
    Hoi, Steven C. H.
    PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2018, : 2660 - 2666
  • [9] Sign Language Classification Using Deep Learning Convolution Neural Networks Algorithm
    Lahari V.R.
    Anusha B.
    Ahammad S.H.
    Immanuvel A.
    Kumarganesh S.
    Thiyaneswaran B.
    Thandaiah Prabu R.
    Amzad Hossain M.
    Rashed A.N.Z.
    Journal of The Institution of Engineers (India): Series B, 2024, 105 (05) : 1347 - 1355
  • [10] Learning with Deep Photonic Neural Networks
    Leelar, Bhawani Shankar
    Shivaleela, E. S.
    Srinivas, T.
    2017 IEEE WORKSHOP ON RECENT ADVANCES IN PHOTONICS (WRAP), 2017,
12345