Spatial awareness application using mixed reality for 3D X-ray CT examination

In the medical field, X-ray computed tomography (CT) is used to determine the size of defects and damage in an examined object, and to diagnose infectious diseases. Generally, data captured by 3D X-ray CT are viewed as images in three directions (sagittal, axial, and coronal) on a computer. However, augmented reality, virtual reality, and mixed reality are emerging as alternatives for imaging captured data for 3D X-ray CT and are used for medical simulations and educational purposes. Although these techniques are capable of 3D expression, they are limited to the representation of the surface structure of the object. The original function required of these technologies is to check the tomographic image of a specific part of the object by specifying any direction and position in three dimensions. This study proposes a method of representation in which a 2D digital imaging communication image is superimposed on a surface-rendered cross-section of an object. In addition, it proposes a pointing system using motion capture and a spatial reality display. The observer can confirm the object from any direction and understand the structure spatially. It can be moved and rotated with movements similar to actually holding, grabbing, moving, and rotating with hands. Moreover, the cross-section can be observed in any direction. In addition, by matching the respective scales of the device and application, the object can be represented with an error of less than 1 mm in the horizontal, vertical, and depth directions with respect to the actual object. Therefore, the proposed method is an effective 3D representation method in 3D X-rays, which are voxel data containing internal information. Furthermore, this method can easily indicate the desired location where the cross-section image can be viewed in a CT image. This study will help improve the efficiency of 3D X-ray inspection and surgery in the medical field.