GPU-based Fast 3D Ultrasound-Endoscope Image Fusion for Complex-Shaped Objects

This paper describes a fast 3D ultrasound-endoscope image fusion technique for complex-shaped objects using GPU-based image processing and multi-planes projection based images mapping. To overcome the small field of view, the lack of image depth and surrounding information of the endoscope, we develop an extended visualization method with the fusion of endoscopic images with a 3-D ultrasound-image. The mosaiced endoscope images are registered to the surface of the 3-D ultrasound model by using a fast GPU-based image rendering method. The 3-D spatial position of the endoscopic images and the ultrasound image are tracked by a 3-D position tracking device for coordination transfer of the two images. Furthermore, we develop a multi-planes projection method for mapping the image onto the complex-shaped objects, especially for the sharp mapping angle. Experimental results show that the system may provide an improved and efficient way of planning and guidance in endoscope based surgical treatment.