The complexity of first-order optimization methods from a metric perspective

被引:0
|
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页数:30
相关论文
共 50 条
  • [31] First-order methods of smooth convex optimization with inexact oracle
    Devolder, Olivier
    Glineur, Francois
    Nesterov, Yurii
    MATHEMATICAL PROGRAMMING, 2014, 146 (1-2) : 37 - 75
  • [32] First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces
    Jordan, Michael I.
    Lin, Tianyi
    Vlatakis-Gkaragkounis, Emmanouil V.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35, NEURIPS 2022, 2022,
  • [33] Monitoring Metric First-Order Temporal Properties
    Basin, David
    Klaedtke, Felix
    Mueller, Samuel
    Zalinescu, Eugen
    JOURNAL OF THE ACM, 2015, 62 (02)
  • [34] First-Order Trotter Error from a Second-Order Perspective
    Layden, David
    PHYSICAL REVIEW LETTERS, 2022, 128 (21)
  • [35] Corrigendum: On the complexity of finding first-order critical points in constrained nonlinear optimization
    C. Cartis
    N. I. M. Gould
    Ph. L. Toint
    Mathematical Programming, 2017, 161 : 611 - 626
  • [36] Complexity of a class of first-order objective-function-free optimization algorithms
    Gratton, S.
    Jerad, S.
    Toint, Ph. L.
    OPTIMIZATION METHODS & SOFTWARE, 2024,
  • [37] Iteration-complexity of first-order augmented Lagrangian methods for convex programming
    Guanghui Lan
    Renato D. C. Monteiro
    Mathematical Programming, 2016, 155 : 511 - 547
  • [38] Iteration-complexity of first-order augmented Lagrangian methods for convex programming
    Lan, Guanghui
    Monteiro, Renato D. C.
    MATHEMATICAL PROGRAMMING, 2016, 155 (1-2) : 511 - 547
  • [39] Iteration complexity analysis of dual first-order methods for conic convex programming
    Necoara, I.
    Patrascu, A.
    OPTIMIZATION METHODS & SOFTWARE, 2016, 31 (03): : 645 - 678
  • [40] Complexity of first-order inexact Lagrangian and penalty methods for conic convex programming
    Necoara, I.
    Patrascu, A.
    Glineur, F.
    OPTIMIZATION METHODS & SOFTWARE, 2019, 34 (02): : 305 - 335