Deep Learning Methods And Applications For Medical Image Registration

Medical image registration is a complicated and ill-posed problem.Designing registration algorithms with high accuracy and high efficiency is a critical problem to be solved for its successful use in clinical imaging assisted diagnosis and treatment.The development of modern medical imaging technologies make image information more complicated and diversified,causing limited performance in the accuracy,efficiency,applicability and robustness for traditional image registration algorithms.Recent studies have found that deep learning can establish effective network models based on rich information from big data and can be effectively used for image registration.This thesis aims to overcome the obstacles in existing medical image registration methods with new deep learning technologies.Specifically,an end-to-end unsupervised affine and deformable cascade registration network model is first designed,and then,regularization methods such as consistency and prior distribution of deformation field are proposed.Finally,based on various clinical and research needs,the proposed deep learning registration methods have been applied in the construction and analysis of longitudinal brain atlases,the analysis of spatial distribution and longitudinal evolution of COVID-19 infection,and the analysis of pulmonary nodule distributions.The main contributions of this thesis are as follows:1.Unsupervised deep learning affine and deformable cascade registration network: In the application of image navigation or image assisted intervention,which requires real-time performance,global registration followed by deformable registration is the standard pipeline.As the present registration algorithms need iterative optimization and take long time,it is difficult to meet the clinical requirement of realtime performance.This thesis proposed an unsupervised affine and deformable cascade registration network.In the first stage,the deep network yields image-level global affine transformation between the input fixed and moving images.In the second stage,voxel-level deformation field is generated,wherein the inputs include the same fixed image as in the first stage and the affine transformed moving image.The two-stage registration network can be trained end-to-end in an unsupervised way by maximizing the global and local normalized correlation coefficients derived from the input images.Thus,no ground-truth anatomical contours or deformation fields drawn/adjusted manually by physicians are needed during training.The cascade registration network can perform image registration directly through forward propagation without any iterative optimization process,and such an inferring process can be completed in less than 1 second.The accuracy Dice of this method in LPBA40,IBSR18,CUMC12 and MGH10 datasets reached 0.71±0.10,0.65±0.23,0.47±0.21 and 0.44±0.19,respectively.2.Unsupervised deep learning symmetric cycle consistency registration network:In clinical tumor adaptive radiotherapy,image registration technology is required to register the image during treatment and the planning computerized tomography(CT)before treatment.The deformation is applied to the propagation and evaluation of the clinical target volume and dose distribution.The inversible,smooth,and consistent deformation field can be realized by performing differential homeomorphic deformation registration,but the calculation is time-consuming.This thesis proposed a symmetric cycle consistent registration network by introducing pair-wise and groupwise deformation consistency regularization.Pair-wise and group-wise deformation consistency are applied in the loss function during training,in addition to traditional image similarity and deformation smoothness regularizations.First,the inverse consistency of an image pair to be registered is introduced through bidirectional registration.Then,the idea of consistency is extended to an image group by applying group consistency.Thus,both pair-wise consistency and group-wise consistency can be used to train the network.The proposed strategy optimizes the training process without introducing any additional network parameters and computational complexity.It can yield robust and consistent results even after switching the order of input images.The deep learning network has similar registration accuracy with the traditional algorithms and yields better deformation consistency compared to other deep registration networks without consistency constraint.Consistency,smoothness and topological correctness of deformation fields are guaranteed.The accuracy Dice in LPBA40,IBSR18,CUMC12 and MGH10 datasets were 0.70±0.05,0.47±0.10,0.48±0.09 and 0.51±0.11,respectively.The consistency Dice was greater than 0.97,and the mean consistency distance was 0.25 mm.The folding number/permillage of deformation fields reached 1204/0.17‰.3.Multi-stage registration network based on deformation prior projected images:Studies on registration of images from different subjects or the same subject with large respiratory movement found that,it is difficult to register the images with large anatomical differences.At the inspiratory and exhalation moments of the lung 4D-CT images,the two images have obvious appearance differences.In the process of atlas construction,robust registration algorithms are required to register images with different morphological differences.Parameters adjustment and iterative optimization are required in traditional algorithms for different applications.Most deep learning algorithms need to set hyperparameters,and the network is not adaptive.The same network cannot be used to process images with different morphological gaps.This thesis proposed a multi-stage registration framework based on the prior distribution of valid deformation fields.The prior image and deformation are generated by the statistical model of group deformation data to enhance the robustness and accuracy of registration.First,the prior distribution of high-dimensional deformation fields is estimated and introduced to the image registration network.During registration,the prior information is used to generate intermediate images which are similar to respective input images.The intermediate images can serve as a bridge for registering the two input images.Then,the registration of the input image pair can be converted to the registration between the generated intermediate image and the corresponding input image using the proposed symmetric cycle consistent registration network.Since the intermediate images are more similar in shape to the corresponding input images,the proposed method can achieve higher accuracy and better consistency,especially between input images with large differences in shape and appearance,and can produce smooth and consistent deformation fields.Experimental results showed that the accuracy Dice of this method is 0.74±0.05,the consistency Dice is 0.94±0.02,the consistency distance is 0.21±0.01 mm,and the deformation field folding degree is0.15‰.4.Hybrid supervision registration network based on prior knowledge of deformation field: To integrate statistical prior constraints of deformations into the registration network training,this thesis proposed a hybrid supervised registration network based on prior knowledge of deformation fields.The method can also be applied into estimating large deformation caused by respiratory movement and into the atlas construction tasks.The specific strategies include:(1)deformation prior variational encoding-decoding data augmentation network and training strategies;(2)wavelet multi-resolution registration network integrating deformation kernel-principal component analysis(k-PCA)prior.On the one hand,a set of deformation fields are generated by the diffeomorphic registration method Sy N.By training the variational encoding and decoding network,the manifold of the deformation fields in the latent space can be obtained,and any valid transformations can be simulated by the decoder via statistical sampling the latent space to serve as the ground-truth for supervised learning of the registration network.Supervised and unsupervised learning are used alternately in the hybrid training.The smoothness and inverse consistency constraints of deformation field are further considered in the network training.On the other hand,a multi-resolution registration network using simple convolutional layers at each resolution level is proposed.The deformation field is solved at the lowest resolution level and refined at higher resolution levels subsequently.The prior distribution of the deformation field is applied to the low-resolution using k-PCA nonlinear modeling.Meanwhile,the image features of the multi-resolution wavelet decomposition are used as the input,which not only preserve the low-frequency image shape but also keep the high-frequency image features.Experimental results showed that the proposed registration networks with hybrid supervision training strategy and the new prior distribution constraint learning have satisfactory performance in registration accuracy,deformation consistency and topological correctness.The mean accuracy Dice of this method was 0.75±0.05,the mean Hausdorff distance was 2.0±0.6mm,the consistency distance was 0.20±0.01 mm,and the deformation field folding degree was 0.16‰.5.Innovative registration applications: The registration methods are applied to medical image analysis,including the construction and analysis of longitudinal brain atlases based on magnetic resonance imaging(MRI)using the multi-stage registration network based on deformation prior projected images,the analysis of COVID-19 infection region distribution and its longitudinal evolution based on CT images,and the analysis of pulmonary nodules distribution based on CT images.A large number of images from different individuals or different time points are registered into respective template image spaces,and the growth or disease distribution patterns of different groups are computed.The changes in the development of organs or tissues,disease progression,and treatment can be seen.It provides prior information for disease early assessment,prediction of disease severity,computer-aided detection and diagnosis,and visual reference for doctors.