EFFICIENT FIRST ORDER METHOD FOR SADDLE POINT PROBLEMS WITH HIGHER ORDER SMOOTHNESS

This paper studies the complexity of finding approximate stationary points for the smooth nonconvex-strongly-concave (NC-SC) saddle point problem: min(x)max(y) f(x,y) . Under the standard first-order smoothness conditions where f is & ell; -smooth in both arguments and mu y -strongly concave in y , existing literature shows that the optimal complexity for first-order methods to obtain an & varepsilon; -stationary point is O(root kappa y & ell;& varepsilon;(-2)) , where kappa(y)=& ell;/mu y is the condition number. However, when Phi(x):=maxyf(x,y) has L2 -Lipschitz continuous Hessian in addition, we derive a first-order algorithm with an O(root kappa y(& ell;1/2)L(2)(1/4)& varepsilon;(-7/4)) complexity by designing an accelerated proximal point algorithm enhanced with the "Convex Until Proven Guilty" technique. Moreover, an improved Omega(kappa y & ell;L-3/7(2)2/7 & varepsilon;(-12/7)) lower bound for first-order method is also derived for sufficiently small & varepsilon; . As a result, given the second-order smoothness of the problem, the complexity of our method improves the state-of-the-art result by a factor of O((& ell;(2) / L-2 & varepsilon;)(1/4)) , while almost matching the lower bound except for a small O((& ell;2L2 & varepsilon;)1/28) factor.