Using curvature information for fast stochastic search

被引：0
|
作者：
Orr, GB
Leen, TK
机构：
来源：
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 9: PROCEEDINGS OF THE 1996 CONFERENCE | 1997年 / 9卷
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We present an algorithm for fast stochastic gradient descent that uses a nonlinear adaptive momentum scheme tea optimize the late time convergence rate. The algorithm makes effective use of curvature information, requires only O(n) storage and computation, and delivers convergence rates close to the theoretical optimum. We demonstrate the technique on linear and large nonlinear back-prop networks.
引用
收藏
页码：606 / 612
页数：7
相关论文
共 50 条
  • [1] A generating set search method using curvature information
    Frimannslund, Lennart
    Steihaug, Trond
    COMPUTATIONAL OPTIMIZATION AND APPLICATIONS, 2007, 38 (01) : 105 - 121
  • [2] A generating set search method using curvature information
    Lennart Frimannslund
    Trond Steihaug
    Computational Optimization and Applications, 2007, 38 : 105 - 121
  • [3] Hitting probabilities for fast stochastic search
    Linn, Samantha
    Lawley, Sean D.
    JOURNAL OF PHYSICS A-MATHEMATICAL AND THEORETICAL, 2024, 57 (30)
  • [4] Information extraction for search engines using fast heuristic techniques
    Hong, Jer Lang
    Siew, Eu-Gene
    Egerton, Simon
    DATA & KNOWLEDGE ENGINEERING, 2010, 69 (02) : 169 - 196
  • [5] Graph based model for information retrieval using a stochastic local search
    Farhi, Sidali Hocine
    Boughaci, Dalila
    PATTERN RECOGNITION LETTERS, 2018, 105 : 234 - 239
  • [6] Hierarchical stochastic search with prior information
    Cremer, N.
    Beckmann, L.
    Chang-Claude, J.
    Conti, D. V.
    GENETIC EPIDEMIOLOGY, 2008, 32 (07) : 684 - 685
  • [7] Manifold Alignment Using Curvature Information
    Mavadati, S. Mohammad
    Mahoor, Mohammad H.
    Zhang, Xiao
    PROCEEDINGS OF 2013 28TH INTERNATIONAL CONFERENCE ON IMAGE AND VISION COMPUTING NEW ZEALAND (IVCNZ 2013), 2013, : 59 - 64
  • [8] Fast semantic object search and detection for vegetable trading information using Steiner tree
    Zhao, Ming
    Tao, Tengyang
    Duanmu, Xiaoyin
    ARTIFICIAL INTELLIGENCE REVIEW, 2014, 41 (03) : 415 - 427
  • [9] Fast semantic object search and detection for vegetable trading information using Steiner tree
    Ming Zhao
    Tengyang Tao
    Xiaoyin Duanmu
    Artificial Intelligence Review, 2014, 41 : 415 - 427
  • [10] Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods
    Yang, Minghan
    Xu, Dong
    Chen, Hongyu
    Wen, Zaiwen
    Chen, Mengyun
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 10649 - 10658
12345