Research On Multi-spectral Fundus Image Processing Algorithm

As one typically non-invasive technique,Multi-Spectral Imaging(MSI)is recently introduced into medical diagnosis and treatment.Since it could produce a sequence of discrete spectral slices that respectively penetrates different light-absorbing chromophores,it is widely accepted as a useful tool for the early identification and recognition of various retinal,optic nerve,and choroidal diseases.However,few studies have paid attention to the processing of MSI images,which would significantly improve the image quality and the accuracy in the diagnosis of ocular lesions for the opthalmologists.It remains a challenge since the relative movement between the humans' eyes and the MSI device during the image acquisition process that might introduce spatial misalignment between MSI slices.Bearing the above-mentioned analysis in mind,we proposed several MSI image processing approaches including image deblurring,registration and segmentation.First of all,the registration of MSI images remains a challenge in practice due to at least two difficulties.On the one hand,the appearances of a set of MSI images significantly differ from each other.This results from the fact that each slice is captured with single wavelength of light,and the fundus tissues at different depths have different reflectivities and absorption to different wavelengths of light.On the other hand,different from the common image registration approaches in which only a pair of images are included,a group of MSI images need to be registered simultaneously.Bearing the above-mentioned analysis in mind,we propose a novel multimodality registration framework for producing a joint alignment of a sequence of MSI slices.The approach is capable of establishing almost optimal matches between image features by leveraging a convex optimization procedure for each sequential MSI image.First,the salient feature points are extracted from each slice of one sequence of input MSI images.The features from the various modalities are mapped to isometric vectors in a common feature space.Meanwhile,the matching cost of all paired images can be measured by spatial-model-based similarity measures.Then,the proposed approach aims to recover globally consistent feature matches through exploiting the similarity measure of all of the image pairs while calculating a predefined motion model.Furthermore,the temporal smoothness of the motion across asequence of MSI images caused by eye movement is taken as a regularization term in the objective function,which is accomplished by employing an alternating minimization strategy.Secondly,for the degeneration caused by motion and out-of-focus blurry effects in MSI images,we presented a multi-modality,multi-image deblurring framework through combining the sequential images after they were aligned simultaneously.Our technique is different from most of the image deblurring techniques in the literature mainly due to the leverage of several unique characteristics of MSI including the spatial and temporal smoothness of the variation in blur kernels and the association between MSI images in each sequence.Notable that resolving the blur artifact in MSI images is not trivial due to two challenges.On the one hand,most of the state-of-the-art image deblurring techniques are proposed for handling single generic image and cannot be directly applied to sequential MSI images except by using a suboptimal way of processing each MSI image independently.On the other hand,it is not straightforward to establish a MSI deblurring technique to process a sequence of slices in MSI simultaneously considering that MSI images are multi-modality and captured sequentially from different depths within humans' eyes.To be specific,the blur kernel in MSI images changes in a smooth fashion spatially and temporally.This smoothness property provides a useful priori knowledge for estimating the blur kernel.Meanwhile,the adjacent images in MSI sequence are correlated because they are captured from retinal/choroidal layers of the same eyeball and various light sources at different wavelengths.We incorporate this correlation in our algorithm to solve the blur kernels simultaneously over all images by measuring image difference with mutual information and by combining our joint inter-modality image alignment algorithm.The proposed method offers at least two main contributions which might help it to impact the field of MSI based diagnosis of eye diseases and other fields using images taken in different spectrum.First,it provides an early tool specifically for deblurring ocular MSI images to our best knowledge.Second,we offer an early solution to eliminating blur artifacts simultaneously from multiple images in different modalities.Thirdly,automated tools are potentially capable of providing more prognostic and predictive markers,which might enable ophthalmologists to assess the aggressiveness of a disease in its early stage or review its response to therapy.In addition,automated andquantitative assessment of the spectral inconsistency is increasingly required along with elevated usage of MSI.Therefore,we propose a method for measuring MSI spectral inconsistency based on an outlier detection framework,which can be used to detect retinal degenerations and segment the corresponding deteriorated regions.Specifically,we measure spectral consistency by extracting the common spectral properties of normal tissues and specify degenerations as outliers that bear inconsistent spectral properties with normal tissues.Mathematically,we define spectral consistency as the fact that the representative features of any pixel in any spectrum can be reconstructed by projecting linearly a unique pixel-specific latent feature vector with a spectrum-specific projection matrix.In contrast,the reconstruction of a spectrally inconsistent pixel requires more than one latent feature vector.The proposed method is founded on a probabilistic Gaussian mixture model and designed to find one maximum a posteriori estimate of the projection matrix and the assignment to the latent feature vector(s)via a stochastic expectation-maximization algorithm.One unique property of this approach lies in the fact that latent feature vectors do not need to be explicitly resolved,leading to a robust and fast estimation.To evaluate the performance of the proposed approaches,we collected several MSI sequences and then conducted comparison experiments on these slices between the state-of-the-art techniques and ours.Experimental results demonstrate that the presented methods not only outperforms the state-of-the-art techniques for deblurring,registration and segmentation in accuracy but also are potentially invaluable for MSI-based ocular disease diagnosis and treatment.