Stereoscopic Augmented Reality for Single Camera Endoscope Using Optical Tracker: A Study on Phantom

Introduction: Endoscopic surgery causes reduced injury to tissues which helps in rapid recovery and less painful post-operative period of patients. However, its narrow field of view and loss of depth perception in endoscope image make surgeon's task difficult. Moreover, in endoscopic surgery surgeons can only see the surface of the surgical anatomy. In this study these limitations have been addressed with the development of an augmented reality system with 3D visualization. Method: The system is comprising of infrared based optical tracker, 2D endoscope system, computer and a 3D monitor. The system was calibrated to bring all the components to a single reference frame of the optical tracker. A phantom and its 3D CT model were used, respectively, to form the real and virtual part of the augmented reality. The endoscope camera position was tracked in 3D using a tracker tool mounted on it. The camera in the virtual environment was updated with the new positions of the endoscope camera. The endoscope video frame and the virtual model rendered image were superimposed to form augmented reality view. The depth map of the virtual scene was further used to make an stereo pair of the augmented reality. Results: The AR system was examined with endoscope video of phantom. It could produce well aligned real and virtual component. The contour different of the two components was 5 to 10 mm. Conclusion: An AR system for endoscopic surgery was successfully implemented on a phantom. It needs to include motion and deformation model to be applied on real patients.