The complexity of first-order optimization methods from a metric perspective

被引:0
|
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页数:30
相关论文
共 50 条
  • [41] General Convergence Analysis of Stochastic First-Order Methods for Composite Optimization
    Necoara, Ion
    JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2021, 189 (01) : 66 - 95
  • [42] Convergence of First-Order Methods for Constrained Nonconvex Optimization with Dependent Data
    Alacaoglu, Ahmet
    Lyu, Hanbaek
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 202, 2023, 202 : 458 - 489
  • [43] Streaming First-Order Optimization Methods With Momentum for Dominant Eigenspace Estimation
    Zhou, Siyun
    NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS, 2025, 32 (01)
  • [44] Stochastic first-order methods for convex and nonconvex functional constrained optimization
    Digvijay Boob
    Qi Deng
    Guanghui Lan
    Mathematical Programming, 2023, 197 : 215 - 279
  • [45] Accelerated First-order Methods for Geodesically Convex Optimization on Riemannian Manifolds
    Liu, Yuanyuan
    Shang, Fanhua
    Cheng, James
    Cheng, Hong
    Jiao, Licheng
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [46] Implementing First-order Optimization Methods: Algorithmic Considerations and Bespoke Microcontrollers
    Zhang, Xinwei
    Sartori, John
    Hong, Mingyi
    Dhople, Sairaj
    CONFERENCE RECORD OF THE 2019 FIFTY-THIRD ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS, 2019, : 296 - 300
  • [47] First-order limits, an analytical perspective
    Nesetril, Jaroslav
    de Mendez, Patrice Ossona
    EUROPEAN JOURNAL OF COMBINATORICS, 2016, 52 : 368 - 388
  • [48] General Convergence Analysis of Stochastic First-Order Methods for Composite Optimization
    Ion Necoara
    Journal of Optimization Theory and Applications, 2021, 189 : 66 - 95
  • [49] First-order approximation methods in reliability-based design optimization
    Chiralaksanakul, A
    Mahadevan, S
    JOURNAL OF MECHANICAL DESIGN, 2005, 127 (05) : 851 - 857
  • [50] Physical Computational Complexity and First-order Logic
    Whyman, Richard
    FUNDAMENTA INFORMATICAE, 2021, 181 (2-3) : 129 - 161