Review of multi-view stereo reconstruction methods based on deep learning

The goal of Multi-view stereo（MVS）Reconstruction is to reconstruct a 3D model of a scene based on a set of multi-view images with known camera parameters，which is a mainstream method of 3D reconstruction in recent years. This paper provides a algorithm evaluation comparison for the latest hundreds of MVS methods based on deep learning. First，we sorted out the existing supervised learning-based MVS methods according to the reconstruction process of feature extraction，cost volume construction，cost volume regularization and depth regression，focusing on the summary of improvement strategies in the two stages of cost volume construction and cost volume regularization. For the unsupervised MVS methods，we mainly analyzed the design of the loss terms of each algorithm. It is classified according to its training mode. Secondly，we summarized the common datasets of MVS methods and their corresponding performance evaluation indexes，and further studied the introduction of strategies such as feature pyramid network，attention mechanism，coarse-to-fine strategy on the performance of MVS networks. In addition，it introduced the specific application scenarios of MVS methods，including digital twin，autonomous driving，robotics，heritage conservation，bioscience and other fields. Finally，we made some suggestions for the improvement direction of MVS methods，and also discussed the future technical difficulties and the research directions of MVS 3D reconstruction. © 2023 Chinese Academy of Sciences. All rights reserved.