LW-DeepFakeNet: a lightweight time distributed CNN-LSTM network for real-time DeepFake video detection

With the large-scale and pervading social media platforms and the recent advances in generative deep learning techniques, it is nowadays quite common to forge highly-realistic and credible misleading videos known as DeepFakes. These videos mean to alter the original intention behind the video to put forth their hidden ploys. In this work, a simple yet effective lightweight time distributed (LW-DeepFakeNet) model that uses both spatial and temporal information to determine whether the video has been altered is proposed. The model utilizes a transfer learning approach with pre-trained convolutional networks for spatial feature extraction, topped up with LSTMs for temporal information extraction, requiring little training data and time. This research also considers a special use case of DeepFake where a particular video sequence has a scene change and proposes a way to counter the class-imbalance present in the dataset. The resulting model is much lighter with up to 152x times reduction in parameter count while achieving a significant accuracy of 99.24% at a remarkable rate of 80 fps.