Self-supervised efficient sample weighting for multi-exit networks

Dynamic sample weighting is an effective approach for improving the inference accuracy and efficiency of different classifiers in multi-exit networks. By bringing the early-exit behavior from testing to training, such a method allows input samples to contribute differently at each sub-network, reducing the gap between training and inference. Recent methods mainly rely on learning a weighting mechanism through reinforcement learning and meta-learning based frameworks jointly with the multi-exit models. However, additional data are usually needed and essential for these approaches, which results in more computation consumption and limits their general applicability. To address these two problems, we propose a Self-supervised Efficient Sample Weighting (SESW) method to predict weights for input samples based on their losses at each exit. We pose the problem as a self-supervised multi-class classification problem. The SESW module is trained to predict if the log-likelihood for the loss of each sample in the current training stage belongs to the most confident classifier. The results of our experiments demonstrate that the proposed method performs comparably to state-of-the-art methods in terms of prediction accuracy and dynamic inference efficiency while achieving a significant improvement in training time.(c) 2023 Elsevier B.V. All rights reserved.