Research On Intelligent Diagnosis Of Macular Disease Based On Multi-modal Retina Imaging

Purpose: To detect the leakage points of central serous chorioretinopathy(CSC)automatically from dynamic images of fundus fluorescein angiography(FFA)using a deep learning algorithm.Methods: The study included 2104 FFA images from 291 FFA sequences of 291 eyes(137 right eyes and 154 left eyes)from 262 patients.The leakage points were segmented with an attention gated network(AGN).The optic disc(OD)and macula region were segmented simultaneously using a U-net.To reduce the number of false positives based on time sequence,the leakage points were matched according to their positions in relation to the OD and macula.Results: With the AGN alone,the number of cases whose detection results perfectly matched the ground truth was only 37 out of 61 cases(60.7%)in the test set.The dice on the lesion level were 0.811.Using an elimination procedure to remove false positives,the number of accurate detection cases increased to 57(93.4%).The dice on the lesion level also improved to 0.949.Conclusions: Our study demonstrated the benefits of using a deep learning algorithm to analyze consecutive FFA images over a temporal sequence to automatically detect leakage points in CSC.The detection performance of the deep learning algorithm was improved with the employment of a FP elimination procedure.This novel finding may pave the way for potential application of artificial intelligence to guide laser therapy and improve the efficiency of clinical practice.Purpose: To develop a deep learning system aiming to grade the severity of epiretinal membrane(ERM)based on optical coherence tomography(OCT)B scans.Methods: In total,3557 OCT images(1883 with ERM,and 1674 normal)of 1593 eyes in 1046 patients are collected at the Eye Center at the Second Affiliated Hospital of Zhejiang University.Three ophthalmologists were trained to label the OCT images according to a classification standard generated by two senior retinal specialists,which divided the images into six categories(normal,stage 1 ~ stage 4,macular hole).3204 OCT images were randomly selected for training and validation,and the rest 353 OCT images were used for testing.Res Net-34 was used as the backbone of the deep learning system,and convolutional block attention module(CBAM)was integrated into each unit of Res Net-34.Evaluate the deep learning system with accuracy,sensitivity,specificity and the area under the receiver operating characteristic curve(AUC).Heatmaps were generated to visualize the procedure of the deep learning system in the classification task.Results: The accuracy of normal,stage 1 ~ stage 4 and macular hole was 0.9773,0.9587,0.9462,0.9207,0.9252 and 0.9332,respectively.The AUC of these six categories was0.9951,0.9285,0.9376,0.9404,0.9574 and 0.9983,respectively.The heatmap of normal images was scattered across the whole retina region.The heatmap of stage 1 ~stage 4 ERM images focused on the fovea and the region below the fovea.The heatmap of macular hole images payed attention to the location of hole as anticipated.Conclusions: This study established an ERM severity grading system with high accuracy,sensitivity and specificity based on deep learning.It may contribute to the clinical practice and scientific research of ERM,and help clinicians to figure out the best operation timing and predict postoperative outcomes.Purpose: To establish a computer aided diagnosis system via deep learning using infrared reflectance(IR)and OCT,realizing the classification of age-related macular degeneration(AMD)into normal,dry AMD and wet AMD.Compare the diagnostic efficacy between single-modality input and multi-modality input.Evaluate the diagnostic power difference between different feature fusing patterns during multi-modality input.Finally,compare the diagnostic performance between junior ophthalmologists and deep learning model.Methods: This is a dual central retrospective study.We retrospectively collected 765 Heidelberg OCT reports(including 2006 pairs of IR & OCT images)at the Eye Center at the Second Affiliated Hospital of Zhejiang University.Two single-modality models(IR_ONLY and OCT_ONLY)and three multi-modality models(OCT_MAIN,IR_MAIN and DUAL)were constructed to compare the diagnostic efficacy.A total of214 Heidelberg OCT reports(including 506 pairs of IR & OCT images)were collected at the Second Affiliated Hospital of Xi’an Jiaotong University,serving as an independent external validation dataset.Three ophthalmologists were invited to do the human-machine comparison under single-modality(given OCT images only)and multi-modality(given paired IR & OCT images)circumstances severally.Results: Models with multi-modality input achieved higher accuracy compared to models with single-modality input,proving that the concomitant use of IR and OCT images improved the classification accuracy for AMD diagnosis.Among the three multi-modality models,OCT_MAIN won the best diagnostic efficacy of classification,with the highest overall accuracy of 0.9608 in test set and 0.9159 in independent external validation dataset.During the human-machine comparison,the model performance is comparable to that of the senior ophthalmologist.Conclusions: The study established a multi-modal intelligent diagnosis system for AMD classification,which was proved to surpass the diagnostic efficacy of single-modality model and match the diagnostic performance of senior ophthalmologist.The combination of multi-modal data enhanced the diagnostic efficacy of deep learning model.It is a valuable complement and optimization to the existing CADx,which reveals a promising application and research future.