Research On Imaging And Fusion Technologies Of Three-dimensional Endoscopes Based On Stereo Cameras With Structured-light

With the development of science and technology,three-dimensional endoscopes are important to improve the success rate of minimally invasive surgery.Using three-dimensional endoscopes to obtain three-dimensional information for surgical scenes and providing it to clinicians can effectively shorten the postoperative recovery period and improve surgical success rate.The paper mainly designed a set of structured-light based 3D scene reconstruction scheme and 3D fusion scheme for stereo electronic endoscope.The constructed three-dimensional endoscope imaging system consists of a structured light generation and illumination module,a stereo endoscope module,an image acquisition,display module,and a back-end processing module.In the paper,the mathematical model and error model of the endoscope camera are defined,and the internal and external parameters of the stereo cameras are obtained by using the OpenCV library calibration to obtain the pose of the rigid endoscope stereo.Aiming at the characteristics of the weak texture in the internal environment,the paper uses three speckle-coded structured-light patterns based on the method of pseudo-random coding.and the semi-global matching algorithm SGBM to obtain the corresponding depth map in a small field of view at close range by the calibrated stereo cameras.In the experiment of measuring the overall depth map,it is realized that all three structured lights can achieve a smoother and complete depth map compared to unstructured light.For almost textureless paper,the minimum error mean of the structured-light depth map of 16×16 symbols and the corresponding fitting plane can reach 0.38mm,and the minimum root mean square error can reach 0.54mm.The speckle spatial structure is used to reduce the parallax search range in the semi-global matching algorithm.Through verification,the accuracy of the depth map is improved and the cost matching time is shortened.The root-mean-square error of the symbol size 4×4 is 0.51 times than the original result,the root-mean-square error of the symbol size 8×8 is 0.57 times than the original result,and the root-mean-square error of the symbol size 16×16 is 0.76 times than the original result.After limiting the parallax,the average single-matching time of the symbol size 4x4 is 0.42 times than the original result,the symbol size 8×8 is 0.37 times than the original result,and the symbol size 16×16 is 0.35 times than the original result.They significantly reduces the time consumed of the average single-matching time.Considering the speed and precision comprehensively,the structured light with the size of 8x8 has the best parallax optimization result after limiting the parallax.Finally,the 3D scene reconstruction is performed on the obtained depth map.Iterative Closest Point(ICP)algorithm is used to achieve 3D fusion of the reconstruction results of the small field of view at a short distance.Finally,it completes the 3D measurement of the kidney model under a large field of view.The field of view of a single picture is 34.03 °,and the field of view after stitching 10 pictures is 2.10 times than the original result,and the field of view is 71.46°,which greatly increases the field of view of observation.The paper mainly explores the encoding of speckle structured light and its optimization of the stereo matching algorithm to achieve 3D scene reconstruction and 3D fusion.Experiments show that the structured-light patterns selected in this paper can improve the precision of the stereo matching algorithm SGBM which is commonly used,and the proposed limited parallax search scheme can further optimize the precision and matching speed of the depth map.The observation field range obtained by 3D fusion has been greatly improved compared to a single image.