Research On Fundus Fluorescence Angiography Images Generation Methods And Application

Fluorescein angiography(FA)can reflect the subtle lesions of retinal and the circulatory state of the retinal vessels,which is regarded as the "gold standard" of retinal-related disease diagnosis.Intravenous injection of sodium fluorescein contrast agent is an essential procedure for FA imaging.However,sodium fluorescein contrast agents are banned for severe hypertension,heart disease,and other patients,and it also has potential adverse effects that can even be life-threatening in severe cases.Therefore,using image generation method to generate the corresponding FA image from the fundus structure image can be a good idea to help the prevention,auxiliary diagnosis,and guidance of retinal-related diseases.For that,this paper first conducts research on the single frame FA image generation,FA sequences generation,and fluorescein leakage detection method.And then the proposed methods are introduced to the two existing retinal imaging systems in the laboratory,one is the line-scanning ophthalmoscope(LSO)which will be industrialized,and the other one is adaptive optics confocal scanning laser ophthalmoscope(AOSLO)which is used for scientific research.The work of this paper can be divided into two parts:method research and method application,of which method research is the core content.In terms of method research,the following three parts of work are mainly carried out:(1)Based on the characteristic that FA imaging is the only ways to evaluate leakage,and to solve the problem that the existing single frame FA image generation method cannot accurately generate retinal vessel details and leakage,a conditional generative adversarial network-based method is proposed,to achieve the auxiliary diagnosis of the retinal-related diseases with fluorescein leakages.A novel local saliency map loss is designed and introduced to facilitate more accurate learning of vessel details and fluorescein leakage features.The proposed method was qualitatively and quantitatively validated on the clinical dataset and publicly available dataset.The experimental results indicate that the proposed method has better performance on the generation of both retinal vessel details and fluorescein leakage structures.Moreover,the average structural similarity between the FA images generated by the proposed method and real FA images can reach 0.8655,which is obviously superior to existing methods.(2)Based on the characteristic that FA imaging can completely observe the dynamic filling process of fluorescent dye in retinal arteries,capillaries,and veins,a novel sequence generative adversarial network is proposed.And it is the first time to achieve the generation of FA sequences in the arterial,venous,and late phases.The proposed method was qualitatively and quantitatively compared with existing one-to-one and one-to-many image translation methods.The experimental results indicate that the proposed model has better performance on the generation of retina vascular,leakage structures,and characteristics of each angiogram phase.In addition,the average structural similarity between the images generated by the proposed method and the real images in each angiogram phase surpass existing supervised one-to-one and one-to-many methods 0.0096 and 0.0413,respectively.(3)Aiming at the problems that the existing intensity-based leakage detection methods have high complexity and long detection time,and the existing supervised learning-based methods require a large amount of manually annotated data,a weakly-supervised learning-based method is proposed.A convolutional block attention module is introduced,and a novel anomaly mask loss is defined based on the class activation map.This facilitates more accurate learning of leakage areas.The proposed method can realize the generation of normal FA images and the quick and accurate detection of fluorescence leakages without the need of labeling the leakage area and paired images.Moreover,the detection speed of the proposed method is more than 20 times faster than the traditional methods,and the average specificity,accuracy,and dice coefficient of the proposed method on the publicly available dataset are higher than the state-of-the-art intensity-based method 0.01.0.05,and 0.03,respectively.As for method application,the proposed method is applied to the two existing retinal imaging systems in the laboratory.The specific work is as follows:(1)To better apply the proposed FA image generation methods to the line-scanning ophthalmoscope(LSO)retinal image,a multi-frame image super-resolution method is proposed.This method was validated on down-sampled and real LSO image sequences,and the results indicate the good performance of the proposed method on retinal vessel enhancement and image quality improvement.The FA image generation method was applied on the images enhanced by the proposed method and the preprocessed images.By comparison,we found that the proposed FA image generation method performs better on the images enhanced by the proposed method.This part of the work can illustrate the application potential of the proposed FA generation method on the clinical-level equipment.(2)To make the proposed FA image generation method has a good application effect on adaptive optics confocal scanning laser ophthalmoscope(AOSLO)retinal vessel image,a conditional generative adversarial network-based method is proposed.Retinal photoreceptor cells are smaller in scale and more sensitive to residual wavefront aberrations,and thus the image blurring is more intense.Therefore,this work first designed a network for the enhancement of the AOSLO image with photoreceptor cells;Then,based on this network,the enhancement of the AOSLO retinal vessel image is realized by using transfer learning technique;Finally,the proposed FA image generation method is applied to the original AOSLO retinal vessel images,the preprocessed images,and the images enhanced by the proposed method.By comparison,we found that the proposed FA image generation method has a better angiographic effect on the AOSLO images enhanced by the proposed method.This experiment can indicate the application potential of the proposed FA generation method on the scientific research-level equipment.In summary,this paper describes the method research on single frame FA image generation,FA sequence generation,and fundus fluorescence leakage detection.The proposed method is applied to the existing clinical-level and scientific research-level equipment in the laboratory.A series of work from the theoretical research to the practical application of the method is realized,which lays the foundation for the subsequent application of the proposed method.