On the Convergence of an Alternating Direction Penalty Method for Nonconvex Problems

This paper investigates convergence properties of scalable algorithms for nonconvex and structured optimization. We consider a method that is adapted from the classic quadratic penalty function method, the Alternating Direction Penalty Method (ADPM). Unlike the original quadratic penalty function method, in which single-step optimizations are adopted, ADPM uses alternating optimization, which in turn is exploited to enable scalability of the algorithm. A special case of ADPM is a variant of the well known Alternating Direction Method of Multipliers (ADMM), where the penalty parameter is increased to infinity. We show that due to the increasing penalty, the ADPM asymptotically reaches a primal feasible point under mild conditions. Moreover, we give numerical evidence that demonstrates the potential of the ADPM for computing local optimal points when the penalty is not updated too aggressively.