Automatic landmark detection and mapping for 2D/3D registration with BoneNet

被引:4
|
作者
Nguyen, Van [1 ]
Pereira, Luis F. Alves F. [1 ,2 ]
Liang, Zhihua [1 ]
Mielke, Falk [1 ,3 ]
Van Houtte, Jeroen [1 ]
Sijbers, Jan [1 ]
De Beenhouwer, Jan [1 ]
机构
[1] Univ Antwerp, Dept Phys, Imec Vis Lab, Antwerp, Belgium
[2] Univ Fed Agreste Pernambuco, Dept Ciencia Comp, Garanhuns, Brazil
[3] Univ Antwerp, Dept Biol, Antwerp, Belgium
关键词
2D; 3D registration; landmark-based registration; pose estimation; automatic landmark detection; deep learning; 2D-3D REGISTRATION; IMAGES;
D O I
10.3389/fvets.2022.923449
中图分类号
S85 [动物医学(兽医学)];
学科分类号
0906 ;
摘要
The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
引用
收藏
页数:17
相关论文
共 50 条
  • [31] Fast DRR generation for 2D/3D registration
    Birkfellner, W
    Seemann, R
    Figl, M
    Hummel, J
    Ede, C
    Homolka, P
    Yang, X
    Niederer, P
    Bergmann, H
    MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION - MICCAI 2005, PT 2, 2005, 3750 : 960 - 967
  • [32] 2D/3D registration algorithm for lung brachytherapy
    Zvonarev, P. S.
    Farrell, T. J.
    Hunter, R.
    Wierzbicki, M.
    Hayward, J. E.
    Sur, R. K.
    MEDICAL PHYSICS, 2013, 40 (02)
  • [33] A framework for automatic creation of gold-standard rigid 3D–2D registration datasets
    Hennadii Madan
    Franjo Pernuš
    Boštjan Likar
    Žiga Špiclin
    International Journal of Computer Assisted Radiology and Surgery, 2017, 12 : 263 - 275
  • [35] Automatic landmark annotation and dense correspondence registration for 3D human facial images
    Jianya Guo
    Xi Mei
    Kun Tang
    BMC Bioinformatics, 14
  • [36] An approach to 2D/3D registration of a vertebra in 2D x-ray fluoroscopies with 3D CT images
    Weese, J
    Buzug, TM
    Lorenz, C
    Fassnacht, C
    CVRMED-MRCAS'97: FIRST JOINT CONFERENCE - COMPUTER VISION, VIRTUAL REALITY AND ROBOTICS IN MEDICINE AND MEDICAL ROBOTICS AND COMPUTER-ASSISTED SURGERY, 1997, 1205 : 119 - 128
  • [37] Automatic landmark annotation and dense correspondence registration for 3D human facial images
    Guo, Jianya
    Mei, Xi
    Tang, Kun
    BMC BIOINFORMATICS, 2013, 14
  • [38] A Unified 3D Mapping Framework Using a 3D or 2D LiDAR
    Zhen, Weikun
    Scherer, Sebastian
    PROCEEDINGS OF THE 2018 INTERNATIONAL SYMPOSIUM ON EXPERIMENTAL ROBOTICS, 2020, 11 : 702 - 711
  • [39] Registration Using Nanotube Stationary Tomosynthesis: Comparison of 3D/3D to 3D/2D Methods
    Frederick, B.
    Lalush, D.
    Chang, S.
    MEDICAL PHYSICS, 2010, 37 (06)
  • [40] 2D/3D-MGR: A 2D/3D Medical Image Registration Framework Based on DRR
    Li, Zhuoyuan
    Ji, Xuquan
    Wang, Chuantao
    Liu, Wenyong
    Zhu, Feiyu
    Zhai, Jiliang
    IEEE ACCESS, 2024, 12 : 124365 - 124374