Fine-Grained Open-Set Deepfake Detection via Unsupervised Domain Adaptation

Deepfake represented by face swapping and face reenactment can transfer the appearance and behavioral expressions of a face in one video image to another face in a different video. In recent years, with the advancement of deep learning techniques, deepfake technology has developed rapidly, achieving increasingly realistic effects. Therefore, many researchers have begun to study deepfake detection research. However, most existing studies on deepfake detection are mainly limited to binary classification of real and fake images, rather than identifying different methods in an open-world scenario, leading to failures in dealing with unknown deepfake categories in practice. In this paper, we propose an unsupervised domain adaptation method for fine-grained open-set deepfake detection. Our method first uses labeled data from the source domain for model pre-training to establish the ability of recognizing different deepfake methods in the source domain. Then, the method uses a Network Memorization based Adaptive Clustering (NMAC) approach to cluster unlabeled images in the target domain and designs a Pseudo-Label Generation (PLG) to generate virtual class labels for unknown deepfake categories by matching the adaptive clustering results with the known deepfake categories in the source domain. Finally, we retrain the initial multi-class deepfake detection model using labeled data of the source domain and pseudo-labeled data of the target domain to improve its generalization ability to unknown deepfake classes presented in the target domain. We validate the effectiveness of the proposed method under multiple open-set fine-grained deepfake detection tasks based on three deepfake datasets (ForgerNet, FaceForensics++, and FakeAVCeleb). Experimental results show that our method has better domain generalization ability than the state-of-the-art methods, and achieves promising performance in fine-grained open-set deepfake detection.