Retinal Image Analysis Based On Generative Adversarial Networks

Retina is the only organ in the human body that one could observe and monitor un-der visible light in a non-invasive manner.Not only could the analysis of retinal images provide early diagnosis and screening of a large number of ophthalmological-related dis-eases,but also could it aid in the deterrence of permanent vision loss as a result of said diseases.In recent years,with excellent feature extraction techniques developed from deep learning,a vast quantity of networks have been employed in the field of retinal im-age analysis where remarkable results have been achieved.Based on such retinal image processing technology,supported by deep learning,more objective diagnostic results can be obtained which would in turn assist ophthalmologists in arriving at a much more informed conclusion.Nevertheless,deep learning based retinal image analysis has its shortcomings-the limited availability of datasets precisely annotated by experts,along with the lack of paired datasets have evidently impeded the application of deep learn-ing in certain research areas within retinal imaging.In this dissertation,based on the unique characteristics of retinal imaging,we constructed three algorithms derived from generative adversarial networks(GANs)and introduced loss functions where applica-ble.These algorithms could lead to a breakthrough in some of the deep learning based retinal image processing techniques.This is being elaborated as follows:(1)We designed and constructed a segmentation algorithm for detecting retinal haemorrhages in newborns using paired GAN for image translation and also proposed a new criterion to grade retinal haemorrhages.Considering the lack of an objective grad-ing criterion to evaluate the severity of retinal haemorrhages,this algorithm utilized paired GAN to simulate a large set of retinal images containing corresponding haemor-rhage annotations,which were used as the training data set for the haemorrhages seg-mentation network.This network can be applied to effectively observe and diagnose haemorrhages,which would in turn overcome a couple of obstacles of having limited access to images with accurately located haemorrhages.This dissertation subsequently defined a new criterion in grading newborn retinal haemorrhages based on the quantita-tive ratio of haemorrhages with respect to the optic disc and the pinpointing of macular region driven by computation that involves its relative proximity to the vascular arcades.(2)We developed a cataract retinal image dehaze algorithm with unpaired GAN to effectively dehaze cataractous images while overcoming the setback of having unpaired datasets.The dehazed image could help ophthalmologists provide a more comprehen-sive treatment plan for cataract patients with other retinal diseases.Via selecting a range of unpaired cataracts to clear retinal images,we designed an unpaired GAN along with two loss functions specifically to address the unique characteristics of retinal images.This network adeptly output cataract-like images while keeping the residing main struc-ture remained,which were then used to train another supervised dehaze network.This algorithm effectively rendered the training of deep learning networks for cataractous images possible even in the absence of paired datasets.Furthermore,using the math-ematically simulated cataract-like images as a prior along with a multi-scale network structure of detection while engaging two proposed loss functions to process retinal images,we had not only profoundly minimized the presence of vessel-like artifacts in the dehazed results,but also enhanced the clarity of the edge of main structures.Two additional no-reference image quality assessment criteria were also introduced to mea-sure the major characteristics of retinal images,which referred to the contrast of main structures against the background and the extent of the presence of vessel-like artifacts.Compared with existing dehaze algorithms,our algorithm not only preserved the out standing dehaze effect,but also vastly diminished the presence of vessel-like artifacts.(3)We proposed a generic super-resolution algorithm for multi-spectral imaging system reconstruction based upon a three-dimensional GAN,which resulted in super-resolution results without altering any light spectrum.We were able to obtain a multi-spectral,three-dimensional data cube by first building a computed tomography imaging spectroscopy system(CTIS).Given the limitation of the low spatial resolution of the reconstruction results,we thereafter designed a generic multi-spectral super-resolution network based on a three-dimensional GAN.This network,taking into account of the reconstructed output of multi-spectral imaging,compellingly merged information of different spectra,produced multi-spectral images of super resolution,and even reduced noise through the integration of three-dimensional convolution,spectral input disorder,and spectral average filtering methods.In order to ensure the consistency of spectral curves before and after having applied super-resolution algorithms,we managed to elim-inate the distortion of spectral angles in the outcome by implementing spectral angle loss functions.This dissertation presented the aforementioned breakthrough research direction that would render CTIS more practical in real-life applications.