Disruption of Object Recognition Systems

In recent times, deep neural networks are being used in a wide variety of applications such as autonomous vehicles, medical imaging and surveillance. While they are becoming increasingly powerful, it is possible to disrupt their task by crafting adversarial inputs. These inputs are essentially perturbations added to the original inputs so that the application using the network, such as an object recognizer, is unable to classify the object in the image. Crafting such inputs to disrupt such a recognition task is termed an adversarial attack. Here, we implement two disruption strategies, Fast Gradient Sign Method (FGSM) and generating perturbations using a generator network. While FGSM requires access to the gradient calculated by the classifier with respect to the input image, the generator trains simultaneously with the classifier network to learn how to craft perturbations. Once the generator network is trained with a particular classifier (say, VGG16), it can disrupt other classifier networks in a black-box fashion as well. Using the same dataset, in this case CIFAR-10, it is possible to adversarially train the classifier to make it more robust to perturbed images. This involves training the classifier on the CIFAR-10 images with both the original images and the ones perturbed by the generator. In experiments, the attack using the generator achieves higher disruption accuracies than FGSM on very deep networks.