RobustMVS: Single Domain Generalized Deep Multi-View Stereo

被引:0
|
作者
Xu, Hongbin [1 ]
Chen, Weitao [2 ]
Sun, Baigui [2 ]
Xie, Xuansong [2 ]
Kang, Wenxiong [1 ,3 ]
机构
[1] South China Univ Technol, Sch Automat Sci & Engn, Guangzhou 510641, Peoples R China
[2] Damo Acad, Alibaba Grp, Hangzhou, Peoples R China
[3] Pazhou Lab, Guangzhou 510335, Peoples R China
基金
中国国家自然科学基金;
关键词
Costs; Task analysis; Three-dimensional displays; Training; Covariance matrices; Visualization; Benchmark testing; Multi-view stereo; domain generalization; deep learning; 3D reconstruction; computer vision; NETWORK;
D O I
10.1109/TCSVT.2024.3399458
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Despite the impressive performance of Multi-view Stereo (MVS) approaches given plenty of training samples, the performance degradation when generalizing to unseen domains has not been clearly explored yet. In this work, we focus on the domain generalization problem in MVS. To evaluate the generalization results, we build a novel MVS domain generalization benchmark including synthetic and real-world datasets. In contrast to conventional domain generalization benchmarks, we consider a more realistic but challenging scenario, where only one source domain is available for training. The MVS problem can be analogized back to the feature matching task, and maintaining robust feature consistency among views is an important factor for improving generalization performance. To address the domain generalization problem in MVS, we propose a novel MVS framework, namely RobustMVS. A Depth-Clustering-guided Whitening (DCW) loss is further introduced to preserve the feature consistency among different views, which decorrelates multi-view features from viewpoint-specific style information based on geometric priors from depth maps. The experimental results further show that our method achieves superior performance on the domain generalization benchmark.
引用
收藏
页码:9181 / 9194
页数:14
相关论文
共 50 条
  • [11] Uncertainty-Aware Deep Multi-View Photometric Stereo
    Kaya, Berk
    Kumar, Suryansh
    Oliveira, Carlos
    Ferrari, Vittorio
    Van Gool, Luc
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, : 12591 - 12601
  • [12] Polarimetric Multi-View Stereo
    Cui, Zhaopeng
    Gu, Jinwei
    Shi, Boxin
    Tan, Ping
    Kautz, Jan
    30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, : 369 - 378
  • [13] Multi-view stereo in the Deep Learning Era: A comprehensive revfiew
    Wang, Xiang
    Wang, Chen
    Liu, Bing
    Zhou, Xiaoqing
    Zhang, Liang
    Zheng, Jin
    Bai, Xiao
    DISPLAYS, 2021, 70
  • [14] Multi-View Stereo: A Tutorial
    Furukawa, Yasutaka
    Hernandez, Carlos
    FOUNDATIONS AND TRENDS IN COMPUTER GRAPHICS AND VISION, 2013, 9 (1-2): : 1 - 148
  • [15] Conditional Single-view Shape Generation for Multi-view Stereo Reconstruction
    Wei, Yi
    Liu, Shaohui
    Zhao, Wang
    Lu, Jiwen
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 9643 - 9652
  • [16] Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo
    Kaya, Berk
    Kumar, Suryansh
    Sarno, Francesco
    Ferrari, Vittorio
    Van Gool, Luc
    2022 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2022), 2022, : 3967 - 3979
  • [17] Review of multi-view stereo reconstruction methods based on deep learning
    Yan H.
    Xu F.
    Huang L.
    Liu C.
    Lin C.
    Guangxue Jingmi Gongcheng/Optics and Precision Engineering, 2023, 31 (16): : 2444 - 2464
  • [18] Multi-view multi-exposure stereo
    Troccoli, Alejandro
    Kang, Sing Bing
    Seitz, Steve
    THIRD INTERNATIONAL SYMPOSIUM ON 3D DATA PROCESSING, VISUALIZATION, AND TRANSMISSION, PROCEEDINGS, 2007, : 861 - 868
  • [19] Deformable convolutions in multi-view stereo
    Masson, Juliano Emir Nunes
    Petry, Marcelo Roberto
    Coutinho, Daniel Ferreira
    Honorio, Leonardo de Mello
    IMAGE AND VISION COMPUTING, 2022, 118
  • [20] Mobile robotic multi-view stereo
    Kumar, Suryansh
    ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2025, 223 : 15 - 27