Convergence in quadratic mean of averaged stochastic gradient algorithms without strong convexity nor bounded gradient

被引：1

作者：

Godichon-Baggioni, Antoine ^{[1
,2
]}

机构：

[1] Sorbonne Univ, Lab Probabil Stat & Modelisat, Paris, France

[2] Sorbonne Univ, Lab Probabil Stat & Modelisat, F-75005 Paris, France

来源：

STATISTICS | 2023年 / 57卷 / 03期

关键词：

Stochastic optimization; stochastic gradient algorithm; averaging; online learning; non-asymptotic convergence; HILBERT-SPACES; DESCENT;

D O I：

10.1080/02331888.2023.2213371

中图分类号：

O21 [概率论与数理统计]; C8 [统计学];

学科分类号：

020208 ; 070103 ; 0714 ;

摘要：

Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high-dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be asymptotically efficient. In this paper, we focus on giving explicit bounds of the quadratic mean error of the estimates, and this, without supposing that the function we would like to minimize is strongly convex or admits a bounded gradient.

引用

页码：637 / 668

页数：32

共 50 条

[21] Strong convergence and bounded perturbation resilience of a modified proximal gradient algorithm
Yanni Guo
Wei Cui
Journal of Inequalities and Applications, 2018
[22] Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms
Lu, Kaihong
Wang, Hongxia
Zhang, Huanshui
Wang, Long
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2024, 69 (04) : 2189 - 2204
[23] The convergence of stochastic gradient algorithms applied to learning in neural networks
Stankovic, S
Tadic, V
AUTOMATION AND REMOTE CONTROL, 1998, 59 (07) : 1002 - 1015
[24] CONJUGATE GRADIENT ALGORITHMS - QUADRATIC TERMINATION WITHOUT LINEAR SEARCHES
DIXON, LCW
JOURNAL OF THE INSTITUTE OF MATHEMATICS AND ITS APPLICATIONS, 1975, 15 (01): : 9 - 18
[25] Modified stochastic gradient identification algorithms with fast convergence rates
Chen, Jing
Ding, Feng
JOURNAL OF VIBRATION AND CONTROL, 2011, 17 (09) : 1281 - 1286
[26] Strong error analysis for stochastic gradient descent optimization algorithms
Jentzen, Arnulf
Kuckuck, Benno
Neufeld, Ariel
von Wurstemberger, Philippe
IMA JOURNAL OF NUMERICAL ANALYSIS, 2021, 41 (01) : 455 - 492
[27] Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
Lin, Junhong
Cevher, Volkan
JOURNAL OF MACHINE LEARNING RESEARCH, 2020, 21
[28] Optimal convergence for distributed learning with stochastic gradient methods and spectral algorithms
Lin, Junhong
Cevher, Volkan
Journal of Machine Learning Research, 2020, 21
[29] On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators
Chen, Changyou
Ding, Nan
Carin, Lawrence
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 28 (NIPS 2015), 2015, 28
[30] Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexity
Du, Simon S.
Hu, Wei
22ND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 89, 2019, 89 : 196 - 205

← 1 2 3 4 5 →