Using curvature information for fast stochastic search

被引：0
|
作者：
Orr, GB
Leen, TK
机构：
来源：
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 9: PROCEEDINGS OF THE 1996 CONFERENCE | 1997年 / 9卷
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We present an algorithm for fast stochastic gradient descent that uses a nonlinear adaptive momentum scheme tea optimize the late time convergence rate. The algorithm makes effective use of curvature information, requires only O(n) storage and computation, and delivers convergence rates close to the theoretical optimum. We demonstrate the technique on linear and large nonlinear back-prop networks.
引用
收藏
页码：606 / 612
页数：7
相关论文
共 50 条
  • [21] A fast non-monotone line search for stochastic gradient descent
    Fathi Hafshejani, Sajad
    Gaur, Daya
    Hossain, Shahadat
    Benkoczi, Robert
    OPTIMIZATION AND ENGINEERING, 2024, 25 (02) : 605 - +
  • [22] A FAST DETERMINATION OF STOCHASTIC EXCITATION WITHOUT CODEBOOK SEARCH IN CELP CODER
    CHOI, JH
    IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, 1995, 3 (06): : 473 - 480
  • [23] A fast non-monotone line search for stochastic gradient descent
    Sajad Fathi Hafshejani
    Daya Gaur
    Shahadat Hossain
    Robert Benkoczi
    Optimization and Engineering, 2024, 25 : 1105 - 1124
  • [24] A Fast Binary Quadratic Programming Solver Based on Stochastic Neighborhood Search
    Lam, Benson Shu Yan
    Liew, Alan Wee-Chung
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (01) : 32 - 49
  • [25] Fast determination of stochastic excitation without codebook search in CELP coder
    DACOM Corp R&D Cent, Daejeon, Korea, Republic of
    IEEE Trans Speech Audio Process, 6 (473-480):
  • [26] Clustering using objective functions and stochastic search
    Booth, James G.
    Casella, George
    Hobert, James P.
    JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 2008, 70 : 119 - 139
  • [27] Feature Selection using Stochastic Diffusion Search
    Alhakbani, Haya
    al-Rifaie, Mohammad Majid
    PROCEEDINGS OF THE 2017 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE (GECCO'17), 2017, : 385 - 392
  • [28] Fast search local extremum for maximal information coefficient (MIC)
    Wang, Shuliang
    Zhao, Yiping
    Shu, Yue
    Yuan, Hanning
    Geng, Jing
    Wang, Shaopeng
    JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2018, 327 : 372 - 387
  • [29] THE SEARCH FOR THE CURVATURE OF SPACE
    SANDAGE, A
    PHYSICA SCRIPTA, 1992, T43 : 7 - 21
  • [30] Heterogeneous Information Network Hashing for Fast Nearest Neighbor Search
    Peng, Zhen
    Luo, Minnan
    Li, Jundong
    Chen, Chen
    Zheng, Qinghua
    DATABASE SYSTEMS FOR ADVANCED APPLICATIONS (DASFAA 2019), PT I, 2019, 11446 : 571 - 586
12345