Medical Image Synthesis Based On Capsule Networks And Deep Generative Adversarial Networks

With the improvement of material living standard,people's life span is extended as a whole,and the fight against disease is also increasing.According to relevant statistics,the probability of a person suffering from cancer in his life is about 60%-70%,and the prevalence rate is higher in the old age.China has 2.6 million cancer cases and 1.8 million deaths every year,and the number is increasing year by year.Therefore,when fighting with the disease,we should not only cultivate a large number of excellent doctors,but also provide them with more effective technical support.Medical imaging has always been the right assistant of doctors,providing a more simple and clear way for treatment.Compared with ordinary image,medical image has more texture,higher resolution,stronger correlation and larger storage space.In order to ensure the reliability of clinical application,people have higher requirements on image preprocessing,image analysis and understanding in compression,segmentation and other aspects.Medical image processing involves computer,mathematics,graphics,medicine and other disciplines.Processing technologies include image transformation,image enhancement,image compression,image smoothing,image segmentation,image recognition,edge sharpening,image fusion,etc.In recent years,researchers have introduced many methods in many fields in medical image processing.After continuous improvement,in the effect and efficiency,the medical image processing algorithm has been improved to a certain extent.There are many ways to obtain medical images,each with its advantages and disadvantages.Sometimes,because of the cost and side effects on human body,we will use the image obtained by other imaging technology to synthesize the image we need.This is an important research direction of medical image synthesis.At present,the traditional method to solve the problem of medical image synthesis is to use the data collected by the imaging system to get through algorithm reconstruction.The methods to improve the quality of medical image focus on improving the efficiency of data acquisition and reconstruction algorithm.The traditional methods can perform well in the scenes with more medical image data and easy to obtain,but they can not perform well in the complex situations such as low resolution,low contrast and less sample size.Moreover,due to the radiation exposure in medical treatment,image verification of different modes is more necessary.In order to solve these problems,this paper proposes to use capsule network and depth generation countermeasure network to solve them.The main research work includes the following parts:1.The image synthesis of arterial spin labeling(ASL)based on improved CapsNet was designed.The most important feature of capsule network is that it can keep the valuable details in the medical image.We use the improved capsule network to synthesize the arterial spin labeling image from the structural MRI image.The important contributions of this study include:(1)When optimizing the structure of capsule network model,three important issues of capsule network,including basic convolution layer,capsule layer and capsule capacity,were studied in depth;(2)ASL images were successfully synthesized from structural magnetic resonance(MRI)images for the first time by using the improved capsule network.2.Locally-constrained WGAN-GP ensemble is designed for ASL image synthesis.Considering that the effect of using the capsule network to improve the similarity is not very obvious,and the effect will become worse when the number of capsule layers is increased,the WGAN-GP integration based on local constraints in this design can get more image details by adding local constraints in the integration.In the qualitative experiment of difference with gold standard,WGAN-GP integrated in all the contrast models to get the lowest difference image,which shows that the image synthesized by WGAN-GP is the closest to gold standard.In addition,WGAN-GP model also achieved the most accurate diagnostic accuracy of 70.43%in the quantitative experiment of dementia diagnosis combined with multimodal method.Experimental results show that WGAN-GP integration has better ASL image synthesis performance than other models.3.WGAN-GP model based on Gaussian mixture model(GMM)noise is designed to synthesize ASL image.In order to better reflect the common non-uniformity characteristics in medical images,the GMM Based Model noise generated by glow model is introduced into WGAN-GP model,which is conducive to the generation of high-quality comprehensive medical images.The model can not only synthesize ASL images from structural MRI images,but also structural MRI images from ASL images.Generally speaking,the resolution of structural MRI image is higher than that of ASL image,so it is more challenging to synthesize structural MRI image from ASL image.Through a series of strict qualitative and quantitative experiments,it is proved that WGAN-GP based on GMM noise can synthesize the structural MRI(or ASL)image closest to the gold standard in all the contrast models.Meanwhile,the average diagnostic accuracy of WGAN-GP which is closest to the gold standard in the diagnosis of dementia is 66.96%(for the synthetic structural MRI image)and 65.75%(for the synthetic ASL image).