Registration of multi-modal images under a complex background combining multiscale features extraction and semantic segmentation

被引:1
|
作者
Jiang, Wenjun [1 ,2 ]
Wu, Ji [1 ,2 ]
Chen, Chi [1 ,2 ]
Chen, Jianming [1 ,2 ]
Zeng, Xiang Jin [1 ,2 ]
Zhong, Liyun [1 ,2 ]
DI, Jianglei [1 ,2 ]
Wu, Xiaoyan [3 ]
Qin, Yuwen [1 ,2 ]
机构
[1] Guangdong Univ Technol, Inst Adv Photon Technol, Sch Informat Engn, Guangzhou 510006, Peoples R China
[2] Guangdong Univ Technol, Guangdong Prov Key Lab Informat Photon Technol, Guangzhou 510006, Peoples R China
[3] China Acad Engn Phys, Inst Fluid Phys, Mianyang 621900, Peoples R China
基金
中国国家自然科学基金;
关键词
SYSTEM; SIFT;
D O I
10.1364/OE.465214
中图分类号
O43 [光学];
学科分类号
070207 ; 0803 ;
摘要
Multi-modal imaging technology has a very broad application value in target recognition and other fields, and image registration is one of its key technologies. In this paper, a multi-modal image registration algorithm that combines multiscale features extraction and semantic segmentation is proposed to achieve accurate registration of polarized images and near-infrared images under complex backgrounds. A classical convolutional neural network ResNet is employed to capture the robust feature descriptors, and a convolutional neural network with an attention mechanism is trained to filter out the irrelevant feature points. Further, the two multi-modal images can be further registered. The experimental results show the feasibility and effectiveness of the proposed method.
引用
收藏
页码:35596 / 35607
页数:12
相关论文
共 50 条
  • [1] Combining Sparse and Dense Features to Improve Multi-Modal Registration for Brain DTI Images
    Moldovanu, Simona
    Toporas, Lenuta Pana
    Biswas, Anjan
    Moraru, Luminita
    ENTROPY, 2020, 22 (11) : 1 - 16
  • [2] Semantic Segmentation of Defects in Infrastructures through Multi-modal Images
    Shahsavarani, Sara
    Lopez, Fernando
    Ibarra-Castanedo, Clemente
    Maldague, Xavier P., V
    THERMOSENSE: THERMAL INFRARED APPLICATIONS XLVI, 2024, 13047
  • [3] Multi-modal semantic image segmentation
    Pemasiri, Akila
    Kien Nguyen
    Sridharan, Sridha
    Fookes, Clinton
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2021, 202
  • [4] EISNet: A Multi-Modal Fusion Network for Semantic Segmentation With Events and Images
    Xie, Bochen
    Deng, Yongjian
    Shao, Zhanpeng
    Li, Youfu
    IEEE TRANSACTIONS ON MULTIMEDIA, 2024, 26 : 8639 - 8650
  • [5] Multi-modal background-aware for defect semantic segmentation with limited data
    Shan, Dexing
    Zhang, Yunzhou
    Liu, Shitong
    JOURNAL OF INTELLIGENT MANUFACTURING, 2024,
  • [6] What Image Features Are Useful for Tumor Segmentation in Multi-Modal Images?
    Hu, Y.
    Grossberg, M.
    Mageras, G.
    MEDICAL PHYSICS, 2015, 42 (06) : 3213 - 3213
  • [7] Segmentation-guided multi-modal registration of liver images for dose estimation in SIRT
    Tang, Xikai
    Rangraz, Esmaeel Jafargholi
    Heeren, Richard's
    Coudyzer, Walter
    Maleux, Geert
    Baete, Kristof
    Verslype, Chris
    Gooding, Mark J.
    Deroose, Christophe M.
    Nuyts, Johan
    EJNMMI PHYSICS, 2022, 9 (01)
  • [8] Segmentation-guided multi-modal registration of liver images for dose estimation in SIRT
    Xikai Tang
    Esmaeel Jafargholi Rangraz
    Richard’s Heeren
    Walter Coudyzer
    Geert Maleux
    Kristof Baete
    Chris Verslype
    Mark J. Gooding
    Christophe M. Deroose
    Johan Nuyts
    EJNMMI Physics, 9
  • [9] Multi-modal and Multi-spectral Registration for Natural Images
    Shen, Xiaoyong
    Xu, Li
    Zhang, Qi
    Jia, Jiaya
    COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 : 309 - 324
  • [10] Pseudo Multi-Modal Approach to LiDAR Semantic Segmentation
    Kim, Kyungmin
    SENSORS, 2024, 24 (23)