Research On 3D Reconstruction Method Of Dynamic Soft Tissue Based On Deep Learning

Since the concept of heartbeat synchronization was proposed,efficient and stable real-time vision-assisted soft tissue surface reconstruction methods in robot-assisted min-imally invasive surgery have been an important research component.The difficulties in accurate reconstruction mainly lie in dynamic blurring and instrument occlusion in en-doscopic reconstruction,and the reconstruction of the intracavity environment needs to effectively overcome the challenges of sparse texture features and inconsistent luminos-ity.The difficulty in achieving real-time reconstruction is that the iterative optimization method used requires a large amount of iteration time.To this end,this thesis further explores more methods that can optimize the whole reconstruction process based on the use of generative models to accomplish dynamic soft tissue 3D reconstruction,mainly as follows:1.In this thesis,a predictive-generative model is proposed for efficiently performing the disparity estimation task of stereo endoscopic images.The model is able to improve the disparity estimation efficiency in the scenes of periodic motion and achieve the real-time reconstruction requirement.In the training process,the pseudo-disparity ground truth is generated using⁺-StyleGAN,and the predictor P is supervised trained based on this.In the application process,the input binocular endoscopic images are mapped to the low-dimensional feature space using the mapping sub-network,and the current potential vector initial value₄^-19)4))is predicted together with the historical frame encoding vector^-1.The experimental results show that the method has higher accuracy,robustness in dynamic soft tissue surface reconstruction tasks,and takes less than 0.4 seconds for single-frame reconstruction.2.For the lack of ground truth in endoscopic video data,this thesis proposes StyleGAN-D,a video generative model that can generate RGB-D video data.based on StyleGAN-V,this thesis switches generator to StyleGAN3,which solves the coordinate adhesion prob-lem.Then the video contextual content is learned and understood using the temporal shift module to obtain continuous jump-free video results.Finally,motion continuity is en-hanced by smoothing motion encoding vectors using the B-sample model.The synthetic stereo endoscopic video dataset is synthesized by interpolating RGB-D dataset.The ex-perimental results show that the video results generated by StyleGAN-D are more realistic and continuous.In addition,this thesis validates the predictive-generative model on the synthetic dataset.The experimental results are consistent with the real dataset,which val-idates the reliability of the synthetic dataset and proves the effectiveness of the proposed disparity estimation method.