TFDEPTH: SELF-SUPERVISED MONOCULARDEPTH ESTIMATION WITH MULITI-SCALE SELECTIVE TRANSFORMER FEATURE FUSION

被引:0
|
作者
Hu, Hongli [1 ]
Miao, Jun [1 ,2 ]
Zhu, Guanghu [1 ]
Yan, Je [2 ]
Chu, Jun [3 ]
机构
[1] Nanchang Hangkong Univ, Sch Aeronaut Mfg Engn, Nanchang, Peoples R China
[2] Chinese Acad Sci, Key Lab Lunar & Deep Space Explorat, Beijing, Peoples R China
[3] Nanchang Hangkong Univ, Key Lab Jiangxi Prov Image Proc & Pattern Recognit, Nanchang 330063, Peoples R China
来源
IMAGE ANALYSIS & STEREOLOGY | 2024年 / 43卷 / 02期
关键词
monocular depth estimation; multi-scale fusion; self-supervised learning; transformer;
D O I
10.105566/ias.2987
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Existing self -supervised models for monocular depth estimation suffer from issues such as discontinuity, blurred edges, and unclear contours, particularly for small objects. We propose a self -supervised monocular depth estimation network with multi -scale selective Transformer feature fusion. To preserve more detailed features, this paper constructs a multi -scale encoder to extract features and leverages the self -attention mechanism of Transformer to capture global contextual information, enabling better depth prediction for small objects. Additionally, the multi -scale selective fusion module (MSSF) is also proposed, which can make full use of multi -scale feature information in the decoding part and perform selective fusion step by step, which can effectively eliminate noise and retain local detail features to obtain a clear depth map with clear edges. Experimental evaluations on the KITTI dataset demonstrate that the proposed algorithm achieves an absolute relative error (Abs Rel) of 0.098 and an accuracy rate (delta) of 0.983. The results indicate that the proposed algorithm not only estimates depth values with high accuracy but also predicts the continuous depth map with clear edges.
引用
收藏
页码:139 / 149
页数:11
相关论文
共 50 条
  • [41] Multi-view Self-supervised Learning and Multi-scale Feature Fusion for Automatic Speech Recognition
    Zhao, Jingyu
    Li, Ruwei
    Tian, Maocun
    An, Weidong
    NEURAL PROCESSING LETTERS, 2024, 56 (04)
  • [42] Multi Self-Supervised Pre-Finetuned Transformer Fusion for Better Vehicle Detection
    Zheng, Juwu
    Ren, Jiangtao
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2025, 22 : 2075 - 2089
  • [43] Multi Self-Supervised Pre-Finetuned Transformer Fusion for Better Vehicle Detection
    Zheng, Juwu
    Ren, Jiangtao
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2025, 22 : 2075 - 2089
  • [44] Multimodal Emotion Recognition With Transformer-Based Self Supervised Feature Fusion
    Siriwardhana, Shamane
    Kaluarachchi, Tharindu
    Billinghurst, Mark
    Nanayakkara, Suranga
    IEEE ACCESS, 2020, 8 (08): : 176274 - 176285
  • [45] Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation
    Chawla, Hemang
    Varma, Arnav
    Arani, Elahe
    Zonooz, Bahram
    2021 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA 2021), 2021, : 5140 - 5146
  • [46] Large Scale Autonomous Driving Scenarios Clustering with Self-supervised Feature Extraction
    Zhao, Jinxin
    Fang, Jin
    Ye, Zhixian
    Zhang, Liangjun
    2021 32ND IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV), 2021, : 473 - 480
  • [47] Self-Supervised Real-World Image Denoising Based on Multi-Scale Feature Enhancement and Attention Fusion
    Tang, Hailiang
    Zhang, Wenxiao
    Zhu, Hailin
    Zhao, Ke
    IEEE ACCESS, 2024, 12 : 49720 - 49734
  • [48] Multi-Sensor Fusion Self-Supervised Deep Odometry and Depth Estimation
    Wan, Yingcai
    Zhao, Qiankun
    Guo, Cheng
    Xu, Chenlong
    Fang, Lijing
    REMOTE SENSING, 2022, 14 (05)
  • [49] Self-supervised memory-guided and attention feature fusion for video anomaly detection
    Jiang, Zitai
    Wang, Chuanxu
    Li, Jiajiong
    Zhao, Min
    Yang, Qingyang
    JOURNAL OF ELECTRONIC IMAGING, 2024, 33 (06)
  • [50] CTFusion: CNN-transformer-based self-supervised learning for infrared and visible image fusion
    Du, Keying
    Fang, Liuyang
    Chen, Jie
    Chen, Dongdong
    Lai, Hua
    Mathematical Biosciences and Engineering, 2024, 21 (07) : 6710 - 6730