The complexity of first-order optimization methods from a metric perspective

被引:0
|
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页数:30
相关论文
共 50 条
  • [21] Complexity parameters for first-order classes
    Arias, M
    Khardon, R
    INDUCTIVE LOGIC PROGRAMMING, PROCEEDINGS, 2003, 2835 : 22 - 37
  • [22] GENERIC COMPLEXITY OF FIRST-ORDER THEORIES
    Rybalov, A. N.
    SIBERIAN ELECTRONIC MATHEMATICAL REPORTS-SIBIRSKIE ELEKTRONNYE MATEMATICHESKIE IZVESTIYA, 2011, 8 : 168 - 178
  • [23] Bounds for the Tracking Error of First-Order Online Optimization Methods
    Madden, Liam
    Becker, Stephen
    Dall'Anese, Emiliano
    JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2021, 189 (02) : 437 - 457
  • [24] Automatic Differentiation of Some First-Order Methods in Parametric Optimization
    Mehmood, Sheheryar
    Ochs, Peter
    INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 108, 2020, 108 : 1584 - 1593
  • [25] Bounds for the Tracking Error of First-Order Online Optimization Methods
    Liam Madden
    Stephen Becker
    Emiliano Dall’Anese
    Journal of Optimization Theory and Applications, 2021, 189 : 437 - 457
  • [26] Avoiding Synchronization in First-Order Methods for Sparse Convex Optimization
    Devarakonda, Aditya
    Demmel, James
    Fountoulakis, Kimon
    Mahoney, Michael W.
    2018 32ND IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS), 2018, : 409 - 418
  • [27] Fast First-Order Methods for Composite Convex Optimization with Backtracking
    Scheinberg, Katya
    Goldfarb, Donald
    Bai, Xi
    FOUNDATIONS OF COMPUTATIONAL MATHEMATICS, 2014, 14 (03) : 389 - 417
  • [28] RELATIVELY SMOOTH CONVEX OPTIMIZATION BY FIRST-ORDER METHODS, AND APPLICATIONS
    Lu, Haihao
    Freund, Robert M.
    Nesterov, Yurii
    SIAM JOURNAL ON OPTIMIZATION, 2018, 28 (01) : 333 - 354
  • [29] Fast First-Order Methods for Composite Convex Optimization with Backtracking
    Katya Scheinberg
    Donald Goldfarb
    Xi Bai
    Foundations of Computational Mathematics, 2014, 14 : 389 - 417
  • [30] First-order methods of smooth convex optimization with inexact oracle
    Olivier Devolder
    François Glineur
    Yurii Nesterov
    Mathematical Programming, 2014, 146 : 37 - 75