Robust Lightweight Neural Network Architecture Search Based on Multi-objective Particle Swarm Optimization

In recent years, lightweight neural network architectures have been improved dramatically in terms of search velocity and accuracy, but it is difficult to maintain their stability and robustness in the face of different attacks. This paper aims to improve the robustness of lightweight neural network architecture. Therefore, a robust lightweight neural network architecture search method based on multi-objective particle swarm (RLNAS-MOPSO) is designed. This method combines adversarial sample training with multi-target search to improve the robustness of the model and speed up the search. Moreover, a novel hierarchical particle updating method based on the designed encoding scheme with lightweight search space is employed to speed up the search process. Last but not least, a new multi-objective evaluation method is proposed to simultaneously evaluate the accuracy and robustness of the model. Resultantly, the accuracy and robustness can be optimized simultaneously by the MOPSO. Experimental results show that the robustness of the obtained architectures by RLNAS-MOPSO can be maintained under different attacks.