CONVERGENCE PROPERTIES OF STOCHASTIC PROXIMAL SUBGRADIENT METHOD IN SOLVING A CLASS OF COMPOSITE OPTIMIZATION PROBLEMS WITH CARDINALITY REGULARIZER

被引：0
|
作者：
Hu, Xiaoyin [1 ]
Liu, Xin [2 ,3 ]
Xiao, Nachuan [4 ]
机构：
[1] Hangzhou City Univ, Sch Comp & Comp Sci, Hangzhou 310015, Peoples R China
[2] Chinese Acad Sci, Acad Math & Syst Sci, State Key Lab Sci & Engn Comp, Beijing, Peoples R China
[3] Univ Chinese Acad Sci, Sch Math Sci, Beijing, Peoples R China
[4] Natl Univ Singapore, Inst Operat Res & Analyt, Singapore, Singapore
来源：
JOURNAL OF INDUSTRIAL AND MANAGEMENT OPTIMIZATION | 2024年 / 20卷 / 05期
基金：
中国国家自然科学基金;
关键词：
Nonsmooth optimization; cardinality regularizer; proximal subgradient method; global convergence; conservative field; NONSMOOTH; GRADIENT; ALGORITHMS; PENALTY;
D O I：
10.3934/jimo.2023149
中图分类号：
T [工业技术];
学科分类号：
08 ;
摘要：
. In this paper, we study a class of composite optimization problems, whose objective function is the summation of a bunch of nonsmooth nonconvex loss functions and a cardinality regularizer. Firstly we investigate the optimality condition of these problems and then suggest a stochastic proximal subgradient method (SPSG) to solve them. Then we establish the almost surely subsequence convergence of SPSG under mild assumptions. We emphasize that these assumptions are satisfied by a wide range of problems arising from training neural networks. Furthermore, we conduct preliminary numerical experiments to demonstrate the effectiveness and efficiency of SPSG in solving this class of problems.
引用
收藏
页码：1934 / 1950
页数：17
相关论文
共 50 条
  • [41] A smoothing Newton method for solving a class of stochastic linear complementarity problems
    Tang, Jia
    Ma, Changfeng
    NONLINEAR ANALYSIS-REAL WORLD APPLICATIONS, 2011, 12 (06) : 3585 - 3601
  • [42] A General Iterative Method for Solving a Certain Class of Optimization Problems
    Bagdasaryan, Armen
    Kadry, Seifedine
    2017 7TH INTERNATIONAL CONFERENCE ON MODELING, SIMULATION, AND APPLIED OPTIMIZATION (ICMSAO), 2017,
  • [43] Iteration Complexity of a Proximal Augmented Lagrangian Method for Solving Nonconvex Composite Optimization Problems with Nonlinear Convex Constraints
    Kong, Weiwei
    Melo, Jefferson G.
    Monteiro, Renato D. C.
    MATHEMATICS OF OPERATIONS RESEARCH, 2023, 48 (02) : 1066 - 1094
  • [44] A computational intelligence method for solving a class of portfolio optimization problems
    Alireza Nazemi
    Narges Tahmasbi
    Soft Computing, 2014, 18 : 2101 - 2117
  • [45] A computational intelligence method for solving a class of portfolio optimization problems
    Nazemi, Alireza
    Tahmasbi, Narges
    SOFT COMPUTING, 2014, 18 (11) : 2101 - 2117
  • [46] DECOMPOSITION METHOD OF SOLVING A CLASS OF COMBINATORIAL OPTIMIZATION PROBLEMS.
    Kravets, V.L.
    Sergienko, I.V.
    1600, (19):
  • [47] On the convergence rate of ordinal optimization for a class of stochastic discrete resource allocation problems
    Dai, LY
    Cassandras, CG
    Panayiotou, CG
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2000, 45 (03) : 588 - 591
  • [48] On the convergence rate of ordinal optimization for a class of stochastic discrete resource allocation problems
    Dai, LY
    Cassandras, CG
    Panayiotou, CG
    PROCEEDINGS OF THE 37TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-4, 1998, : 1692 - 1697
  • [49] On Stochastic Proximal-Point Method for Convex-Composite Optimization
    Nedic, Angelia
    Tatarenko, Tatiana
    2017 55TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2017, : 198 - 203
  • [50] A single cut proximal bundle method for stochastic convex composite optimization
    Liang, Jiaming
    Guigues, Vincent
    Monteiro, Renato D. C.
    MATHEMATICAL PROGRAMMING, 2024, 208 (1-2) : 173 - 208
12345