Integrating 3D and 2D Views of Medical Image Data in Virtual Reality for Efficient Navigation

Medical imaging techniques, e.g., CT or MRI, produce large sets of 2D and 3D data for diagnosis and operative planning. Virtual Reality (VR) could provide a more intuitive and efficient way of interacting with such data. While 3D representations are well suited to show 3D spatial relationships, 2D image slices are important to show details, i.e., both 3D and 2D are required. While researchers have investigated separated approaches for presenting 3D and 2D data in VR, we present a solution that closely integrates 3D and 2D and provides an intuitive and efficient interaction technique for navigating the information by syncing the position of a 3D cursor inside the 3D representation to matching 2D images on different axes. We conducted an empirical evaluation comparing our approach to navigating the data in 2D and found that the VR method was significantly faster, although, at the cost of significantly increased task load.