Surplus-based accelerated algorithms for distributed optimization over directed networks

This paper investigates a distributed optimization problem based on the framework of a multi-agent system over a directed communication network, where the global cost function is the sum of the local cost functions of agents. The communication network is abstracted as a weight-unbalanced directed graph. First, a surplus-based accelerated algorithm with a fixed stepsize (SAAFS) is proposed by integrating the gradient tracking strategy into the surplus-based consensus protocol to address the problem considered. The matrix norm argument and matrix perturbation theory are employed to prove the linear convergence of SAAFS under the assumption that each local cost function is strongly convex with the Lipschitz continuous gradient. Second, the limitation of the stepsize, which is common to all agents, is relaxed in the cases of different stepsizes for each agent, such that the surplus-based accelerated algorithm with an uncoordinated stepsize (SAAUS) is proposed. It is proven that SAAUS also has a linear convergence rate if the upper bound of the uncoordinated stepsize at each agent is restricted by a sufficiently small positive number. Finally, two simulation examples are provided to evaluate the proposed algorithms and illustrate that both SAAFS and SAAUS achieve acceleration, particularly for ill-conditioned optimization problems. (c) 2022 Elsevier Ltd. All rights reserved.