| With the improvement of social living standards and changes in people’s lifestyles,the incidence of diabetes increased year by year.DR(Diabetic Retinopathy),a typical complication of diabetes,has a serious impact on the patient’s vision and even leads to blindness.The number of patients with DR is getting more and more with increasing incidence of diabetes.At present,the main way diagnosing DR is to observe the lesions in the fundus image by doctors.However,in the face of a large number of fundus images,it is not only time-consuming and laborious,but also the diagnostic result is easily affected by the personal factors(experience,fatigue,emotion,etc)of the doctors.Therefore,this paper combining the deep learning of rapid development in the field of computer vision these years studies the automatic diagnosis method of DR,which is of great significance for assisting doctors to diagnose and improving the accuracy and efficiency of diagnosis.The main work of this paper is as follows:Firstly,an image annotation model based on deep learning is studied,which can g generate a natural statement describing the lesion points in the DR fundus image.In order to cope with the problem of insufficient medical images,the model uses transfer learning and uses the Inception-V3 network pretrained on the ImageNet dataset as the model’s feature extractor of image.In order to fully extract the features of the input image,the input image is processed into 7 sub-images,including 6 sub-images for extracting local features and one sub-image for extracting global features.7 sub-images are respectively fed into the Inception-V3 network to extract features,and then all features are merged to obtain the feature vector representing image.In order to get the final text description,LSTM(Long Short-Term Memory)is used to decode the feature vector.The keyword search is used to evaluate the diagnosis result,and experiment results show that the effect is better than directly using CNN(Convolution Neural Network)network to multi-classify.Secondly,a method based on deep learning to generate fundus images is studied.The method uses the vascular vein of the fundus image and the text description of lesions as the constraint conditions to generate fundus image.The input text is encoded by LSTM network to match the corresponding image vector obtained by CNN encoding.The vascular vein is convolved to obtain feature maps of different scales.The encoded text information is merged with the feature maps and then a series of deconvolution operation is used to generate the fundus image with the same size as the input vascular vein.The generated fundus image is discriminated by the discriminant network,and the network parameters are updated by the back propagation algorithm according to the discriminat results,and the generated image by the network gradually approaches the real fundus image.The results show that the algorithm can generate realistic fundus images.Finally,an automatic detection algorithm based on loss weighting for the lesions of DR fundus images has studied.Based on the generative adversarial network,the algorithm can detect four lesions of early DR lesions,including microaneurysms,hemorrhages,hardexudates and softexudates.For the input fundus image,the network simultaneously outputs four detection results,corresponding to four lesions,respectively.In order to make four kinds of lesions with different shapes can produce better results at the same time,the algorithm improves the loss function,and uses different values to weight the lesions areas of different lesions which makes the lesions of different areas play the same role when updating the network parameters.The final experimental results show that updating the model parameters using the weighted loss function can make the model more accurate for the detection of lesions. |
