Adversarial Cross-modal Domain Adaptation for Multi-modal Semantic Segmentation in Autonomous Driving

3D semantic segmentation is a vital problem in autonomous driving. Vehicles rely on semantic segmentation to sense the surrounding environment and identify pedestrians, roads, and other vehicles. Though many datasets are publicly available, there exists a gap between public data and real-world scenarios due to the different weathers and environments, which is formulated as the domain shift. These days, the research for Unsupervised Domain Adaptation (UDA) rises for solving the problem of domain shift and the lack of annotated datasets. This paper aims to introduce adversarial learning and cross-modal networks (2D and 3D) to boost the performance of UDA for semantic segmentation across different datasets. With this goal, we design an adversarial training scheme with a domain discriminator and render the domain-invariant feature learning. Furthermore, we demonstrate that introducing 2D modalities can contribute to the improvement of 3D modalities by our method. Experimental results show that the proposed approach improves the mIoU by 7.53% compared to the baseline and has an improvement of 3.68% for the multi-modal performance.