Surface Reconstruction From Stereo Laparoscopic Images For Augmented Reality-Based Image Guided Surgery

Currently, despite its significant benefits, such as minimal incision diameter, short period of recovery, etc., laparoscopic surgery is limited by problems with visibility, lowering surgeon’s ability to navigate the instruments inside a patient’s organ. This especially stands true in abdominal surgery field, where, unlike in neurosurgery etc., there exists significant movement and deformation of internal organ structure on short enough timescales to make images displayed on a screen at the very least confusing to a surgeon. Modern computer vision technology develops a wide range of approaches that can offer information support to surgeons during such procedure. One such approach is the Augmented Reality, novel field of research employing fast volumetric image processing and 3D model generation to create and display a combined image of real world environment and computer graphic objects (or vice versa).By combining preoperative volumetric image acquired by the means of tomography or ultrasonic scanning with surface recognition techniques from mono-or stereoscopic video it is possible to create a concise compound image with clear outline of internal organs of the abdomen. Consequently, this image can be displayed during surgery on a computer screen, projected image or specialized display-equipped eyewear.This work looks into implementation of a surface recognition and reconstruction algorithm to be used with a complex system of medical imaging and surgical devices to improve quality of laparoscopic image-guided surgery by creating a real-time 3D map of surfaces of internal organs using the visual data feed received from a laparoscopic stereo camera, with consequent display of the said map using the augmented reality based system.Firstly, we analyze existing technical solutions for pre-and intra-operative image acquisition and talk in depth about different technical approaches to endoscopic devices and laparoscopic imagery. Next, we look into possible techniques for augmented reality image display. Then, we discuss existing mathematic camera models and mathematical basis for surface reconstruction and software approaches to surface recognition.In the practical part of this work, we analyze three different constituents of stereoscopic image processing. Firstly, we take a look into calibration procedures and their quantitative qualities for stereoscopic cameras. Then, we assess the possibility of image rectification and disparity calculation without camera calibration by using feature tracking algorithms. Next, we use MATLAB computing environment to create a software module prototype capable of rectifying stereoscopic video feed and, based on created disparity map, generating a point cloud that might be displayed as a surface image and possibly overlaid on an existing 3D-model. Then we analyze the results by measuring efficiency and performance of the said module in processing a publically available laparoscopic surgery stereoscopic video. And lastly we compare efficiency and computational demands of different filtering and outlier removal techniques that might be used to refine the generated point cloud and achieve higher levels of precision while reconstructing the internal organ surfaces.