Review of multi-view stereo reconstruction methods based on deep learning

被引：1
|
作者：
Yan H. [1 ]
Xu F. [1 ]
Huang L. [2 ]
Liu C. [1 ]
Lin C. [1 ]
机构：
[1] School of Science, Jiangxi University of Science and Technology, Ganzhou
[2] School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou
来源：
Guangxue Jingmi Gongcheng/Optics and Precision Engineering | 2023年 / 31卷 / 16期
关键词：
3D reconstruction; deep learning; depth estimation; homography transformation; multi-view stereo;
D O I：
10.37188/OPE.20233116.2444
中图分类号：
学科分类号：
摘要：
The goal of Multi-view stereo（MVS）Reconstruction is to reconstruct a 3D model of a scene based on a set of multi-view images with known camera parameters，which is a mainstream method of 3D reconstruction in recent years. This paper provides a algorithm evaluation comparison for the latest hundreds of MVS methods based on deep learning. First，we sorted out the existing supervised learning-based MVS methods according to the reconstruction process of feature extraction，cost volume construction，cost volume regularization and depth regression，focusing on the summary of improvement strategies in the two stages of cost volume construction and cost volume regularization. For the unsupervised MVS methods，we mainly analyzed the design of the loss terms of each algorithm. It is classified according to its training mode. Secondly，we summarized the common datasets of MVS methods and their corresponding performance evaluation indexes，and further studied the introduction of strategies such as feature pyramid network，attention mechanism，coarse-to-fine strategy on the performance of MVS networks. In addition，it introduced the specific application scenarios of MVS methods，including digital twin，autonomous driving，robotics，heritage conservation，bioscience and other fields. Finally，we made some suggestions for the improvement direction of MVS methods，and also discussed the future technical difficulties and the research directions of MVS 3D reconstruction. © 2023 Chinese Academy of Sciences. All rights reserved.
引用
收藏
页码：2444 / 2464
页数：20
相关论文
共 124 条
  • [11] YAO Y, LUO Z X, Computer Vision-ECCV 2018, pp. 785-801, (2018)
  • [12] LI L Y, JIANG L Y，, Et al., A review on deep learning techniques for cloud detection methodologies and challenges［J］, Signal，Image and Video Processing, 15, 7, pp. 1527-1535, (2021)
  • [13] Deep Learning for Multi-View Stereo via Plane Sweep：a Survey ［EB/OL］, (2021)
  • [14] Multi-view stereo in the deep learning era：a comprehensive review［J］, Displays, 70, (2021)
  • [15] GALLUP D, Et al., Real-time plane-sweeping stereo with multiple sweeping directions［C］, 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, (2007)
  • [16] YAO Y, LUO Z X, Et al., Recurrent MVSNet for high-resolution multi-view stereo depth inference［C］, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）, pp. 5520-5529, (2019)
  • [17] CHEN R, Et al., Point-based multi-view stereo network［C］, 2019 IEEE/CVF International Conference on Computer Vision（IC⁃ CV）, pp. 1538-1547
  • [18] XUE Y Z，, CHEN J S，, WAN W T，, Et al., MVSCRF：learning multi-view stereo with conditional random fields［C］, 2019 IEEE/CVF Inter⁃ national Conference on Computer Vision（ICCV）, pp. 4311-4320, (2019)
  • [19] FAN Z W，, ZHU S Y，, Et al., Cascade cost volume for high-resolution multi-view stereo and stereo matching［C］, 2020 IEEE/CVF Confer⁃ ence on Computer Vision and Pattern Recognition （CVPR）, pp. 2492-2501, (2020)
  • [20] YANG J Y, MAO W, ALVAREZ J M，, Et al., Cost volume pyramid based depth inference for multi-view stereo［C］, 2020 IEEE/CVF Confer⁃ ence on Computer Vision and Pattern Recognition （CVPR）, pp. 4876-4885, (2020)
12345678910