Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation

被引:3
|
作者
Bruggemann, David [1 ]
Sakaridis, Christos [1 ]
Brodermann, Tim [1 ]
Van Gool, Luc [1 ]
机构
[1] Swiss Fed Inst Technol, Zurich, Switzerland
来源
2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023) | 2023年
关键词
D O I
10.1109/ICCV51070.2023.01045
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Standard unsupervised domain adaptation methods adapt models from a source to a target domain using labeled source data and unlabeled target data jointly. In model adaptation, on the other hand, access to the labeled source data is prohibited, i.e., only the source-trained model and unlabeled target data are available. We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain. The target set consists of unlabeled pairs of adverse- and normalcondition street images taken at GNSS-matched locations. Our method-CMA-leverages such image pairs to learn condition-invariant features via contrastive learning. In particular, CMA encourages features in the embedding space to be grouped according to their condition-invariant semantic content and not according to the condition under which respective inputs are captured. To obtain accurate cross-domain semantic correspondences, we warp the normal image to the viewpoint of the adverse image and leverage warp-confidence scores to create robust, aggregated features. With this approach, we achieve state-of-the-art semantic segmentation performance for model adaptation on several normal-to-adverse adaptation benchmarks, such as ACDC and Dark Zurich. We also evaluate CMA on a newly procured adverse-condition generalization benchmark and report favorable results compared to standard unsupervised domain adaptation methods, despite the comparative handicap of CMA due to source data inaccessibility. Code is available at https://github.com/brdav/cma.
引用
收藏
页码:11344 / 11353
页数:10
相关论文
共 50 条
  • [31] On the Robustness of Semantic Segmentation Models to Adversarial Attacks
    Arnab, Anurag
    Miksik, Ondrej
    Torr, Philip H. S.
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020, 42 (12) : 3040 - 3053
  • [32] On Adversarial Robustness of Point Cloud Semantic Segmentation
    Xu, Jiacen
    Zhou, Zhe
    Feng, Boyuan
    Ding, Yufei
    Li, Zhou
    2023 53RD ANNUAL IEEE/IFIP INTERNATIONAL CONFERENCE ON DEPENDABLE SYSTEMS AND NETWORKS, DSN, 2023, : 531 - 544
  • [33] Benchmarking the Robustness of LiDAR Semantic Segmentation Models
    Yan, Xu
    Zheng, Chaoda
    Xue, Ying
    Li, Zhen
    Cui, Shuguang
    Dai, Dengxin
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2024, 132 (07) : 2674 - 2697
  • [34] Confidence-Aware Contrastive Learning for Semantic Segmentation
    Lv, Lele
    Liu, Qing
    Kan, Shichao
    Liang, Yixiong
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 5584 - 5593
  • [35] Region-aware Contrastive Learning for Semantic Segmentation
    Hu, Hanzhe
    Cui, Jinshi
    Wang, Liwei
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 16271 - 16281
  • [36] SELF-SEMANTIC CONTOUR ADAPTATION FOR CROSS MODALITY BRAIN TUMOR SEGMENTATION
    Liu, Xiaofeng
    Xing, Fangxu
    El Fakhri, Georges
    Woo, Jonghye
    2022 IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING (IEEE ISBI 2022), 2022,
  • [37] Prototypical Contrastive Learning for Domain Adaptive Semantic Segmentation
    Liu, Quansheng
    Pu, Chengdao
    Gao, Fang
    Yu, Jun
    2023 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN, 2023,
  • [38] Unsupervised domain adaptation for semantic segmentation via cross-region alignment
    Wang, Zhijie
    Liu, Xing
    Suganuma, Masanori
    Okatani, Takayuki
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2023, 234
  • [39] Condition-Invariant Semantic Segmentation
    Sakaridis, Christos
    Bruggemann, David
    Yu, Fisher
    Van Gool, Luc
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2025, 47 (04) : 3111 - 3125
  • [40] Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime
    Dai, Dengxin
    Van Gool, Luc
    2018 21ST INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2018, : 3819 - 3824