Multi-spectral transformer with attention fusion for diabetic macular edema classification in multicolor image

Diabetic macular edema (DME) is a common cause of vision-threatening diseases. Multicolor image (MCI) enables the diagnosis of DME by providing multiple spectral images of fundus structures. However, the accuracy of existing machine learning methods is still low as they fail to exploit the characteristics of MCI. A multi-spectral vision transformer model with an attention fusion (Atfusion) module is proposed in this paper for DME classification. The transformer extracts the global features of the image using a self-attentive mechanism. In addition, a novel fusion technique - AtFusion module is created to efficiently fuse the multi-spectral features from both branches. We examine the empirical performance of the proposed algorithm on our in-house data sets. The classifier is able to predict the DME status of MCIs with accuracy of 0.951, sensitivity of 0.931, specificity of 0.953, and AUC of 0.933. The experimental results prove that the proposed methodology achieves relatively better performance than the state-of-the-art method.