The Research On Predicting And Generating Thyroid Elastography Images Based On Generative Adversarial Network

The thyroid nodule is a common clinical disease,with a prevalence of over 20%among adults.But 80-90%of these nodules are benign and require surgery merely for high clinical suspicion of malignancy.Ultrasound Elastography(USE)has played a vital role in improving the accuracy of non-invasive detection of malignant thyroid nodules since conventional ultrasound cannot be accurately used to determine the benignity of nodules.However,USE requires specialised equipment and long-term clinical experience;hence the elastography images and scores are highly subjective and lack objective quantification methods,which makes large errors.Generative Adversarial Network(GAN)has opened up innovative ideas for image translation,driving new high levels of generation quality and diversity.In medical imaging,an increasing number of studies are using GAN to generate high-quality images.USE can be regarded as the image translation task in GAN,i.e.the generation of the corresponding elastography image from the ultrasound image.However,the existing image translation networks(e.g.pix2pix,pix2pixHD,and LPTN)produce problems such as inadequate extraction of nodule features,inaccurate color distribution,and inability to predict strain ultrasound information.Based on those characteristics and problems above,this paper studies the Strain Elastography(SE)and proposes an elastography image prediction generation network and automatic elastography scoring method relying on attention mechanism,feature fusion,and multi-scale ultrasound prediction.This work effectively reduces manual interference and therefore domains high clinical practical value.The main contents of this paper are the following four parts.(1)Construction of thyroid elastography dataset.Considering that there is currently no public dataset of elastography images and the collected data cannot be used directly,this paper cooperates with the Sixth People’s Hospital of Shanghai to collect elastography data from 2019 to 2021 for a total of 579 patients.An automatic preprocessing method for the ultrasound data was proposed in conjunction under expert instruction.The method includes fixed position cutting and image gradient algorithm cutting.Furthermore,the cropped data were classified,sorted,and annotated to construct the corresponding 726 sets of the"strain ultrasound+elastography" dataset,512 sets of the"non-strain ultrasound+strain ultrasound+elastography" dataset,and 635 sets of the"ultrasound segmentation" dataset.(2)Elastography image generation network based on attention mechanism and feature fusion.To address the problems of insufficient nodule feature extraction and inaccurate color distribution in elastography images generated by existing image translation networks,this paper combines the characteristics of thyroid elastography images and proposes a generative network(Ultrasound Elastography-GAN,USE-GAN)for generating elastography images from strain ultrasound data by adopting the nodule location attention module,the color channel attention module,the feature fusion module,and the color distribution loss function.Experimental results show that USE-GAN generates high-quality images(PSNR=28.719,SSIM=0.469,MSE=87.340,FID=94.312).And the results achieve an accuracy reaching 84.62%in elastography scoring,which are 13.59%better than the best existing image translation methods.(3)Non-strain ultrasound prediction and elastography image generation network based on multi-scale ultrasound prediction.To address the problem that existing image translation networks cannot complete the ultrasound image prediction,resulting in low quality of the generated images,this paper decouples the generator of GAN into two sub-networks:a multi-scale ultrasound prediction sub-network and an elastography generation sub-network.This GAN also adds a cross-network feature fusion module and a new intermediate layer bootstrap loss function to enhance image quality.Therefore,the proposed Multi-Scale Predicting and Generating network(MSPG)is put forward to predict the strain ultrasound image features and obtain the final elastography results.The comparison experiments show that the MSPG could generate higher quality elastography images(PSNR=28.399,SSIM=0.356,MSE=94.086,FID=103.622),and the accuracy of elastography scoring reached 79.26%,which is 69.32%before the best existing image translation methods.The ablation experiment results prove the effectiveness of each improved module.(4)Automatic scoring method for thyroid nodules concerning lesion segmentation.To address the problems of subjectivity and inconsistent quantification standards in elastography scoring by clinicians,this paper firstly studies the automatic segmentation of thyroid nodules.Moreover,an automatic elastography scoring method for thyroid nodules is designed and implemented,including data pre-processing,ultrasound prediction,elastography image generation,and mapping of the hardened proportion based on color recognition.The accuracy over expert scoring reached 89.36%.So this method can play an effective supporting role in actual clinical diagnosis and research work and has high clinical practical value.