Automatic landmark detection and mapping for 2D/3D registration with BoneNet

被引:4
|
作者
Nguyen, Van [1 ]
Pereira, Luis F. Alves F. [1 ,2 ]
Liang, Zhihua [1 ]
Mielke, Falk [1 ,3 ]
Van Houtte, Jeroen [1 ]
Sijbers, Jan [1 ]
De Beenhouwer, Jan [1 ]
机构
[1] Univ Antwerp, Dept Phys, Imec Vis Lab, Antwerp, Belgium
[2] Univ Fed Agreste Pernambuco, Dept Ciencia Comp, Garanhuns, Brazil
[3] Univ Antwerp, Dept Biol, Antwerp, Belgium
关键词
2D; 3D registration; landmark-based registration; pose estimation; automatic landmark detection; deep learning; 2D-3D REGISTRATION; IMAGES;
D O I
10.3389/fvets.2022.923449
中图分类号
S85 [动物医学(兽医学)];
学科分类号
0906 ;
摘要
The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
引用
收藏
页数:17
相关论文
共 50 条
  • [41] Mental registration of 2D and 3D visualizations (An empirical study)
    Tory, M
    IEEE VISUALIZATION 2003, PROCEEDINGS, 2003, : 371 - 378
  • [42] LiftReg: Limited Angle 2D/3D Deformable Registration
    Tian, Lin
    Lee, Yueh Z.
    Estepar, Raul San Jose
    Niethammer, Marc
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION, MICCAI 2022, PT VI, 2022, 13436 : 207 - 216
  • [43] 2D/3D image registration using regression learning
    Chou, Chen-Rui
    Frederick, Brandon
    Mageras, Gig
    Chang, Sha
    Pizer, Stephen
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2013, 117 (09) : 1095 - 1106
  • [44] Robust 2D/3D Calibration Using RANSAC Registration
    Fortenbury, Billy Ray
    Guerra-Filho, Gutemberg
    ADVANCES IN VISUAL COMPUTING, ISVC 2012, PT I, 2012, 7431 : 179 - 188
  • [45] An Augmented Reality Approach for Initializing 2D/3D Registration
    Gong, Ren Hui
    Guler, Ozgur
    Lovejoy, John
    Yaniv, Ziv
    AUGMENTED REALITY ENVIRONMENTS FOR MEDICAL IMAGING AND COMPUTER-ASSISTED INTERVENTIONS, 2013, 8090 : 117 - 126
  • [46] Evaluation of similarity measures for 3D/2D image registration
    Skerl, Darko
    Likar, Bostjan
    Pernus, Franjo
    MEDICAL IMAGING 2006: IMAGE PROCESSING, PTS 1-3, 2006, 6144
  • [47] 3D/2D Image Registration for Microsphere Tracking in Radioembolization
    Henry, E. Courtney
    Syme, Alasdair
    Mawko, George
    MEDICAL PHYSICS, 2019, 46 (11) : 5391 - 5392
  • [48] 2D/3D registration with the CMA-ES method
    Gong, Ren Hui
    Abolmaesumi, Purang
    MEDICAL IMAGING 2008: VISUALIZATION, IMAGE-GUIDED PROCEDURES, AND MODELING, PTS 1 AND 2, 2008, 6918
  • [49] 2D/3D rigid registration by integrating intensity distance
    Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou
    215163, China
    不详
    130033, China
    不详
    100049, China
    不详
    100069, China
    Guangxue Jingmi Gongcheng, 10 (2815-2824):
  • [50] Initial Pose Estimation Method in 2D/3D Registration
    Sun, Tao
    Guo, Ke
    Liu, Chuanba
    Zhang, Tao
    Song, Yimin
    Ma, Xinlong
    Tianjin Daxue Xuebao (Ziran Kexue yu Gongcheng Jishu Ban)/Journal of Tianjin University Science and Technology, 2022, 55 (02): : 143 - 150