Assess and Guide: Multi-modal Fake News Detection via Decision Uncertainty

The prevalence of fake news, in both visual and textual forms, has become a critical and urgent issue. Existing multi-modal fake news detection methods aim to improve detection performance by leveraging information from different modalities and integrating it into a unified, comprehensive representation. However, challenges arise when different modalities hold conflicting beliefs. This suggests the existence of information asymmetry and imbalance between various modalities, which has not been adequately recognized in previous work. In this regard, focusing on the uncertainty of each modality facilitates a more nuanced understanding of its reliability and informativeness. In this work, we distinguish among uni-modal information sources through accurate estimation of their decision uncertainty, employing this uncertainty as a confidence measure and indicator of information imbalance to guide feature fusion. In particular, 1) To provide accurate uncertainty estimation, an uncertainty learning method based on the Dirichlet distribution is employed; 2) To fuse multi-modal information, uncertainty is used as a guiding factor to govern the contribution of each modality. Extensive experiments on two real-world datasets Weibo and Twitter validate the effectiveness of the proposed method.