Research On 2D/3D Medical Image Rigid Registration Methods Based On Deep Learning

Two-dimensional to three-dimensional medical imaging registration represents one of the enabling technologies for medical imaging-based techniques,which aims to align the pre-operative high-resolution 3D data and the intra-operative 2D data in the same coordinate system.It can be used to provide complementary information on pathology and anatomy from diverse modal sources to assist in accurate diagnosis and advanced image guidance,etc.Promoting the accuracy and execution efficiency of the 2D/3D medical image registration is highly valuable in improving the precision of clinical treatment and diagnosis,as well as reducing the prognosis time.In recent years,deep learning based 2D/3D image registration has attracted much attention from researchers.Compared with the traditional 2D/3D medical image registration algorithm,the trained model can directly predict the registration parameters in a single forward pass,leading to great improvements in both computational efficiency and accuracy.Nevertheless,there are several limitations in the learning-based methods,such as the infeasiblity in obtaining and labeling the large amount of clinical data,low training efficiency and multimodal registration problem,etc.,which hinder their further development.Besides,the geometric relationship between the 2D image and the registration parameters is not fully explored,and the non-differentiable process of traditional image generation limits the further improvement of the model performance.To handle these problems,the research work of this dissertation can be mainly summarized from the perspectives of data generation,training efficiency,accuracy and robustness for deep learning-based 2D/3D medical image registration,which is listed as follows:(1)To solve the problem that traditional generation methods cannot differentiate the image-to-parameter process,based on the imaging principle,the differentiable algorithm for projection and slice imaging is proposed,respectively.In these two differentiable rendering algorithms,both the X-ray path and slice imaging planes are regarded as a thick continuum,in which their adjacent pixels contribute to the generation of projection and slice images with decaying weights in distance.In this way,it causes more continuous change in the image gradient.With the imaging quality preserved,both the differentiable projection imaging and the differentiable slice imaging contain more 3D voxel information,which enables the analytical gradient flow from the image domain to the registration parameter space more effectively,thus enhancing the performance of the model.It cannot only be used for offline sample generation,but can also be employed to improve the performance of traditional optimized registration methods.(2)To address the problem of the deficiencies of existing methods in terms of training efficiency,computational speed and stability,a differentiable resampling-based network for real-time slice-to-volume registration is proposed.In the method,a deep network is trained to establish the direct mapping from image appearance to out-of-plane transformation parameters,and is invariant to the remaining in-plane parameters through online data augmentation,thus reducing parameter dimension with the training sample size dropped significantly.Moreover,based on the idea of differentiable rendering in the research I,a differentiable resampling module for the negative feedback learning process of CNN is introduced to further improve the model performance.Besides,an enhanced image structure similarity operator is applied to better handle the multi-modal problem where the silhouettes between the multi-modal images are different to some extent.Extensive experiments show that,compared with existing deep learning methods,the proposal can effectively improve the training efficiency,accuracy and speed of 2D/3D medical image registration.(3)For the problem of multi-modal registration,the enhanced image structure similarity operator in the research II cannot address the large modal difference in silhouettes.To handle this,an unsupervised cross-modality domain adaptation network is proposed for the projection-to-volume registration.The proposed method first trains a model on simulated data to establish the appearance-pose relationship,and then uses adversarial learning to adapt the model to the X-ray image domain.The domain gap is significantly narrowed by alignment in both pixel and feature space synergistically.Besides,the entire training process does not require annotating X-ray images,and can relieve the difficulty of labeling clinical data,which is highly valuable in practice.Experiments demonstrate the feasibility and superiority of UCMDAN over other existing domain adaptation methods for multi-modal gap in 2D/3D medical image registration.In summary,this dissertation studies 2D/3D medical image rigid registration methods based on deep learning,and proposes new algorithms to handle two registration situations and the problems of existing methods,which improve the registration accuracy and generalization ability.The research work of this dissertation will provide theoretical and technical support for further development of related clinical applications.