TFDEPTH: SELF-SUPERVISED MONOCULARDEPTH ESTIMATION WITH MULITI-SCALE SELECTIVE TRANSFORMER FEATURE FUSION

被引:0
|
作者
Hu, Hongli [1 ]
Miao, Jun [1 ,2 ]
Zhu, Guanghu [1 ]
Yan, Je [2 ]
Chu, Jun [3 ]
机构
[1] Nanchang Hangkong Univ, Sch Aeronaut Mfg Engn, Nanchang, Peoples R China
[2] Chinese Acad Sci, Key Lab Lunar & Deep Space Explorat, Beijing, Peoples R China
[3] Nanchang Hangkong Univ, Key Lab Jiangxi Prov Image Proc & Pattern Recognit, Nanchang 330063, Peoples R China
来源
IMAGE ANALYSIS & STEREOLOGY | 2024年 / 43卷 / 02期
关键词
monocular depth estimation; multi-scale fusion; self-supervised learning; transformer;
D O I
10.105566/ias.2987
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Existing self -supervised models for monocular depth estimation suffer from issues such as discontinuity, blurred edges, and unclear contours, particularly for small objects. We propose a self -supervised monocular depth estimation network with multi -scale selective Transformer feature fusion. To preserve more detailed features, this paper constructs a multi -scale encoder to extract features and leverages the self -attention mechanism of Transformer to capture global contextual information, enabling better depth prediction for small objects. Additionally, the multi -scale selective fusion module (MSSF) is also proposed, which can make full use of multi -scale feature information in the decoding part and perform selective fusion step by step, which can effectively eliminate noise and retain local detail features to obtain a clear depth map with clear edges. Experimental evaluations on the KITTI dataset demonstrate that the proposed algorithm achieves an absolute relative error (Abs Rel) of 0.098 and an accuracy rate (delta) of 0.983. The results indicate that the proposed algorithm not only estimates depth values with high accuracy but also predicts the continuous depth map with clear edges.
引用
收藏
页码:139 / 149
页数:11
相关论文
共 50 条
  • [1] Self-Supervised Monocular Depth Estimation Based on Full Scale Feature Fusion
    Wang C.
    Chen Y.
    Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics, 2023, 35 (05): : 667 - 675
  • [2] Using full-scale feature fusion for self-supervised indoor depth estimation
    Deqiang Cheng
    Junhui Chen
    Chen Lv
    Chenggong Han
    He Jiang
    Multimedia Tools and Applications, 2024, 83 : 28215 - 28233
  • [3] Using full-scale feature fusion for self-supervised indoor depth estimation
    Cheng, Deqiang
    Chen, Junhui
    Lv, Chen
    Han, Chenggong
    Jiang, He
    MULTIMEDIA TOOLS AND APPLICATIONS, 2024, 83 (09) : 28215 - 28233
  • [4] Geometrized Transformer for Self-Supervised Homography Estimation
    Liu, Jiazhen
    Li, Xirong
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 9522 - 9531
  • [5] SFT: Few-Shot Learning via Self-Supervised Feature Fusion With Transformer
    Lim, Jit Yan
    Lim, Kian Ming
    Lee, Chin Poo
    Tan, Yong Xuan
    IEEE ACCESS, 2024, 12 : 86690 - 86703
  • [6] Indoor self-supervised monocular depth estimation based on level feature fusion
    Cheng D.
    Zhang H.
    Kou Q.
    Lü C.
    Qian J.
    Guangxue Jingmi Gongcheng/Optics and Precision Engineering, 2023, 31 (20): : 2993 - 3009
  • [7] Multimodal Image Fusion via Self-Supervised Transformer
    Zhang, Jing
    Liu, Yu
    Liu, Aiping
    Xie, Qingguo
    Ward, Rabab
    Wang, Z. Jane
    Chen, Xun
    IEEE SENSORS JOURNAL, 2023, 23 (09) : 9796 - 9807
  • [8] Self-Supervised Surround-View Depth Estimation with Volumetric Feature Fusion
    Kim, Jung-Hee
    Hur, Junhwa
    Tien Phuoc Nguyen
    Jeong, Seong-Gyun
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [9] Multi-Frame Self-Supervised Depth Estimation with Multi-Scale Feature Fusion in Dynamic Scenes
    Zhong, Jiquan
    Huang, Xiaolin
    Yu, Xiao
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 2553 - 2563
  • [10] A Self-Supervised Transformer With Feature Fusion for SAR Image Semantic Segmentation in Marine Aquaculture Monitoring
    Fan, Jianchao
    Zhou, Jianlin
    Wang, Xinzhe
    Wang, Jun
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61