Improving Adversarial Robustness of Deep Neural Networks via Linear Programming

Adversarial training provides an effective means to improve the robustness of neural networks against adversarial attacks. The nonlinear feature of neural networks makes it difficult to find good adversarial examples where project gradient descent (PGD) based training is reported to perform best. In this paper, we build an iterative training framework to implement effective robust training. It introduces the Least-Squares based linearization to build a set of affine functions to approximate the nonlinear functions calculating the difference of discriminant values between a specific class and the correct class and solves it using LP solvers by simplex methods. The solutions found by LP solvers turn out to be very close to the real optimum so that our method outperforms PGD based adversarial training, as is shown by extensive experiments on the MNIST and CIFAR-10 datasets. Especially, our methods can provide considerable robust networks on CIFAR-10 against the strong strength attacks, where the other methods get stuck and do not converge.