Automatic landmark detection and mapping for 2D/3D registration with BoneNet

被引:4
|
作者
Nguyen, Van [1 ]
Pereira, Luis F. Alves F. [1 ,2 ]
Liang, Zhihua [1 ]
Mielke, Falk [1 ,3 ]
Van Houtte, Jeroen [1 ]
Sijbers, Jan [1 ]
De Beenhouwer, Jan [1 ]
机构
[1] Univ Antwerp, Dept Phys, Imec Vis Lab, Antwerp, Belgium
[2] Univ Fed Agreste Pernambuco, Dept Ciencia Comp, Garanhuns, Brazil
[3] Univ Antwerp, Dept Biol, Antwerp, Belgium
关键词
2D; 3D registration; landmark-based registration; pose estimation; automatic landmark detection; deep learning; 2D-3D REGISTRATION; IMAGES;
D O I
10.3389/fvets.2022.923449
中图分类号
S85 [动物医学(兽医学)];
学科分类号
0906 ;
摘要
The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Robust registration of 2D and 3D point sets
    Fitzgibbon, AW
    IMAGE AND VISION COMPUTING, 2003, 21 (13-14) : 1145 - 1153
  • [22] Retinal image registration from 2D to 3D
    Lin, Yuping
    Medioni, Gerard
    2008 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOLS 1-12, 2008, : 2830 - 2837
  • [23] AUTOMATED 2D/3D REGISTRATION FOR LUNG BRACHYTHERAPY
    Zvonarev, P.
    Farrell, T.
    Hunter, R.
    Hayward, J.
    Wierzbicki, M.
    Sur, R.
    RADIOTHERAPY AND ONCOLOGY, 2011, 99 : S479 - S479
  • [24] Interactive initialization of 2D/3D rigid registration
    Gong, Ren Hui
    Gueler, Oezguer
    Kuerklueoglu, Mustafa
    Lovejoy, John
    Yaniv, Ziv
    MEDICAL PHYSICS, 2013, 40 (12)
  • [25] An ASEKF Algorithm for 2D and 3D Radar Registration
    Liu Jianfeng
    Zuo Yan
    Xue Anke
    2013 32ND CHINESE CONTROL CONFERENCE (CCC), 2013, : 4758 - 4761
  • [26] 2D/3D registration of neonatal brain images
    Vansteenkiste, Ewout
    Vandemeulebroucke, Jef
    Philips, Wilfried
    BIOMEDICAL IMAGE REGISTRATION, PROCEEDINGS, 2006, 4057 : 272 - 279
  • [27] Advancements in 2D/3D Image Registration Methods
    Hjouj, Fawaz
    Jouini, Mohamed Soufiane
    Al-Khaleel, Mohammad
    IEEE ACCESS, 2023, 11 : 34698 - 34708
  • [28] Rotation invariance principles in 2D/3D registration
    Birkfellner, W
    Wirth, J
    Burgstaller, W
    Baumann, B
    Staedele, H
    Hammer, B
    Gellrich, NC
    Jacob, AL
    Regazzoni, P
    Messmer, P
    MEDICAL IMAGING 2003: VISUALIZATION, IMAGE-GUIDED PROCEDURES, AND DISPLAY, 2003, 5029 : 807 - 814
  • [29] 2D/3D registration based on volume gradients
    Wein, W
    Röper, B
    Navab, N
    MEDICAL IMAGING 2005: IMAGE PROCESSING, PT 1-3, 2005, 5747 : 144 - 150
  • [30] 2D and 3D nonrigid body registration in fMRI
    Singh, M
    AlDayeh, L
    Patel, P
    1996 IEEE NUCLEAR SCIENCE SYMPOSIUM - CONFERENCE RECORD, VOLS 1-3, 1997, : 1474 - 1478