Image Quality Assessment Of Breast Ultrasound Based On Multi-task Learning

Under the background of medical-engineering integration,ultrasonic autonomous scanning robots have attracted more and more attention of researchers because of their advantages such as reducing repetitive physical labor of doctors,non-direct contact between doctors and patients,and remote diagnosis and treatment.One of the key technologies of ultrasonic autonomous scanning robots is the automatic quality assessment of ultrasonic images.The value of automatic quality assessment of ultrasonic images lies in the real-time quality assessment of the ultrasonic images collected by the robot,and the generation of feedback signals to guide the robot’s visual servo control system to respond accordingly.The action in turn ensures the imaging quality of the robotic system.Whether the ultrasound image can be evaluated correctly determines the quality of the acquired images of the ultrasound autonomous scanning robot.Existing robotic systems usually use pixel-level feature statistical methods such as grayscale and confidence maps,but clinicians’ evaluation of ultrasound image quality not only depends on pixel grayscale values,but also on image content.Therefore,the existing ultrasonic image quality methods are not accurate enough to carry out objective quality evaluation.In order to make the ultrasound image quality assessment method closer to the clinical judgment process and more in line with the actual clinical needs,in this study,the deep learning method was introduced into the quality evaluation of medical breast ultrasound images.The main contents of this study are as follows:（1）This thesis proposes a global ultrasound image quality assessment method based on bilinear convolutional neural network.In the assessment of breast ultrasound image quality,there is a small difference between images of different quality levels,but a large difference between images of the same quality level.This is similar to the image features in the fine-grained classification task,so the bilinear convolutional neural network（BCNN）in the fine-grained classification task is introduced into the ultrasound quality evaluation task,and the BCNN module is used to analyze the extracted features.High-order feature coding fusion improves the accuracy of quality evaluation.In addition,there is a serious imbalance of samples in the quality level distribution of breast ultrasound images,which is alleviated by the weighted sampling method in this study.Through the combination of the BCNN method and the weighted sampling method,the results obtained by the global ultrasound image quality assessment method proposed in this study are consistent with the doctor’s manual annotation results,reaching 0.842 PLCC（Pearson Linear Correlation Coefficient）,which is better than the traditional method.Confidence map method.（2）This thesis proposes a multi-task learning-based method for classification and segmentation of breast ultrasound images.The global image quality assessment method has a large deviation for the ultrasound image containing the lesion area.This is because the quality of the ultrasound image containing the lesion area depends more on the imaging state of the lesion area,because the image quality of the lesion area determines the quality of the ultrasound image.The diagnostic value of the whole image.In order to determine whether the breast ultrasound image contains a lesion area,and to extract the lesion area image,a breast ultrasound image classification network based on transfer learning and a breast ultrasound image lesion segmentation network based on transfer learning are used in this thesis.In the segmentation task,the performances of the popular segmentation network models U-Net,PSPNet and Deeplab v3+ in the field of medical image processing are compared in the task of breast ultrasound lesion segmentation.Aiming at the problem of scarcity of data and labels in the field of breast ultrasound image analysis,a multi-task learning method is introduced to share the weights of feature extractors,improve the generalization ability of the model,and improve the accuracy of model classification and image segmentation.After using the multi-task learning method,the accuracy of the classification model for breast ultrasound classification reached 95.2%,and the accuracy of the segmentation model for the classification of breast lesions reached0.920 Dice（dice coefficient）.（3）This thesis proposes a local reduced-reference ultrasound image quality assessment method based on the segmentation results in the lesion area.The problem that has always plagued medical image quality evaluation,especially the image quality evaluation of the lesion area,is that there is a lack or no high-quality reference image,and the image data of the lesion area is small,so it is difficult to use the deep learning method that requires a large amount of data.Aiming at this problem,this research creatively proposes to transform the original non-reference image quality evaluation problem into a reduced-reference image quality evaluation problem by introducing the segmentation mask obtained in the segmentation task.Taking the mask obtained from segmentation as the reference image,four local image quality assessment indicators of the lesion area were designed and used in this study: lesion area contrast,structural similarity,MSE and PSNR to characterize the image semantic structure and image pixel statistics to gain the similarity between the local image of the lesion area and the segmentation mask.In this way,the design of the quality evaluation index also has a strong interpretability of the algorithm.In order to obtain the results of image quality assessment in the lesion area,the four quality assessment indicators and the doctor’s subjective quality assessment results of the corresponding images are input into the multilayer perceptron for training,so that the multilayer perceptron has the ability to evaluate the local image quality of breast ultrasound.In the case of the existing segmentation accuracy,the consistency between the quality evaluation result of the local image of the lesion area and the doctor’s subjective evaluation result is 0.749 PLCC.If the segmentation accuracy is high enough,the consistency between the evaluation results of this method and the doctor’s subjective evaluation results can reach 0.897 PLCC.