A2FSeg: Adaptive Multi-modal Fusion Network for Medical Image Segmentation

Magnetic Resonance Imaging (MRI) plays an important role in multi-modal brain tumor segmentation. However, missing modality is very common in clinical diagnosis, which will lead to severe segmentation performance degradation. In this paper, we propose a simple adaptive multi-modal fusion network for brain tumor segmentation, which has two stages of feature fusion, including a simple average fusion and an adaptive fusion based on an attention mechanism. Both fusion techniques are capable to handle the missing modality situation and contribute to the improvement of segmentation results, especially the adaptive one. We evaluate our method on the BraTS2020 dataset, achieving the state-of-the-art performance for the incomplete multi-modal brain tumor segmentation, compared to four recent methods. Our A2FSeg (Average and Adaptive Fusion Segmentation network) is simple yet effective and has the capability of handling any number of image modalities for incomplete multi-modal segmentation. Our source code is online and available at https://github.com/Zirui0623/A2FSeg.git.