The complexity of first-order optimization methods from a metric perspective

被引:0
|
作者
Lewis, A. S. [1 ]
Tian, Tonghua [1 ]
机构
[1] Cornell Univ, ORIE, Ithaca, NY 14850 USA
基金
美国国家科学基金会;
关键词
Nonsmooth optimization and first-order algorithms; Slope; KL property; Complexity; Semi-algebraic; PROXIMAL POINT ALGORITHM; DESCENT METHODS; ERROR-BOUNDS; LOJASIEWICZ INEQUALITIES; GRADIENT FLOWS; CONVERGENCE; MINIMIZATION; SPACES;
D O I
10.1007/s10107-024-02091-2
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
A central tool for understanding first-order optimization algorithms is the Kurdyka-& Lstrok;ojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather "slope", a purely metric notion. By highlighting this view, and avoiding any use of subgradients, we present a simple and concise complexity analysis for first-order optimization algorithms on metric spaces. This subgradient-free perspective also frames a short and focused proof of the KL property for nonsmooth semi-algebraic functions.
引用
收藏
页数:30
相关论文
共 50 条
  • [1] The complexity of first-order optimization methods from a metric perspective
    Lewis, Adrian S.
    Tian, Tonghua
    arXiv, 2023,
  • [2] First-Order Methods for Convex Optimization
    Dvurechensky, Pavel
    Shtern, Shimrit
    Staudigl, Mathias
    EURO JOURNAL ON COMPUTATIONAL OPTIMIZATION, 2021, 9
  • [3] On the Iteration Complexity of Oblivious First-Order Optimization Algorithms
    Arjevani, Yossi
    Shamir, Ohad
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 48, 2016, 48
  • [4] Optimal complexity and certification of Bregman first-order methods
    Dragomir, Radu-Alexandru
    Taylor, Adrien B.
    D'Aspremont, Alexandre
    Bolte, Jerome
    MATHEMATICAL PROGRAMMING, 2022, 194 (1-2) : 41 - 83
  • [5] Optimal complexity and certification of Bregman first-order methods
    Radu-Alexandru Dragomir
    Adrien B. Taylor
    Alexandre d’Aspremont
    Jérôme Bolte
    Mathematical Programming, 2022, 194 : 41 - 83
  • [6] FIRST-ORDER PENALTY METHODS FOR BILEVEL OPTIMIZATION
    Lu, Zhaosong
    Mei, Sanyou
    SIAM JOURNAL ON OPTIMIZATION, 2024, 34 (02) : 1937 - 1969
  • [7] Control Interpretations for First-Order Optimization Methods
    Hu, Bin
    Lessard, Laurent
    2017 AMERICAN CONTROL CONFERENCE (ACC), 2017, : 3114 - 3119
  • [8] From error bounds to the complexity of first-order descent methods for convex functions
    Bolte, Jerome
    Trong Phong Nguyen
    Peypouquet, Juan
    Suter, Bruce W.
    MATHEMATICAL PROGRAMMING, 2017, 165 (02) : 471 - 507
  • [9] From error bounds to the complexity of first-order descent methods for convex functions
    Jérôme Bolte
    Trong Phong Nguyen
    Juan Peypouquet
    Bruce W. Suter
    Mathematical Programming, 2017, 165 : 471 - 507
  • [10] A General Framework for Decentralized Optimization With First-Order Methods
    Xin, Ran
    Pu, Shi
    Nedic, Angelia
    Khan, Usman A.
    PROCEEDINGS OF THE IEEE, 2020, 108 (11) : 1869 - 1889