Image Processing Strategies And Optimization Based On Saliency Model Under Simulated Prosthetic Vision

Blindness is the most serious disability that affects the quality of human life.Among the many blinding diseases,age-related macular degeneration(AMD)and retinitis pigmentosa(RP)are the major incurable retinal degenerative blinding diseases.At present,retinal prosthesis has become an effective approach for visual function rehabilitation of blind patients with retinal degeneration diseases.However,due to the constraints in manufacturing technology,materials,and biosafety,the implantable electrode number is limited.Although some research teams are currently working on the development of high-density retinal prosthesis,the number of electrodes developed is still inadequate compared to millions of ganglion cells and billion-grade photoreceptor cells.In addition,the neural regulation mechanism of color perception induced by electrically stimulating retinal neurons is still unknown,so it is difficult for the retinal prosthesis to produce controlled color vision.At the same time,a large number of clinical trials have shown that the brightness level percepted by the prosthesis recipients is also very limited.Therefore,current prosthesis recipients only obtain low-resolution,low-brightness,and color-lost prosthetic vision perception.It results in the lack of most of the feature information such as texture,contrast,and spatial depth information in the recipients' visual experience.As a result,prosthesis recipients lose the ability of visual selective attention that human normal vision relies on these salient visual features.Meanwhile,due to the restrictions from retinal concave structure and surgical safety,the size of the electrode array cannot be too large,which covers macular area in retina,providing only prosthetic visual percepts with small visual field for recipients.Since an external camera and a video processing unit are an important part of retina prostheses,the visual experience of prosthesis recipients can be improved by image optimization processing.Based on this,to solve the above key issues,the thesis carried out image processing strategies and optimization based on saliency model under simulated prosthetic vision.Specific research includes the following three aspects:1)Aiming at most loss of salient visual information in the current low-resolution prosthetic vision and the deficiency of the saliency segmentation algorithm currently available in prosthetic vision,In the thesis,a novel saliency segmentation algorithm was proposed,and two prosthetic vision optimization strategies based on saliency segmentation were further proposed to effectively extract and enhance the objects of interest in everyday scenes.The results from the psychophysical experiment based on object recognition task under simulated prosthesis vision showed that the proposed segmentation algorithm and prosthetic vision optimization strategies could effectively extract the objects of interest in living scenes and significantly improve the ability of subjects to recognize objects under low-resolution prosthetic vision.In addition,the results found that subjects' recognition performance was also positively affected by segmentation results,the uniqueness of objects,and the paired-interrelated objects in the scene.2)Although various sophisticated image processing algorithms proposed by researchers have been shown to improve the visual perception of prosthesis recipients and raise their ability to complete visual tasks,most of them cannot achieve real-time processing due to the complexity of the proposed algorithms and the limited platform processing power.This greatly curbs the practical application of these algorithms on retinal prosthesis implantation system.Considering the superiority of the saliency model on prosthetic vision and the real-time requirement of image processing algorithms when in practical application,we proposed a novel real-time saliency detection algorithm,aiming to quickly and effectively detect foreground objects in a scene.The experiment results on the two public image sets showed that the proposed algorithm outperformed 4 existing real-time saliency object detection algorithms in terms of several widely accepted indicators.Based on the method,a real-time image optimization strategy was proposed to improve low-resolution prosthetic vision perception,given the fact that prosthesis recipients have almost lost the ability of visual saliency detection.Results demonstrated by verification of conducting two eye-hand-coordination visual tasks that under simulated prosthetic vision,the proposed strategy could significantly improve the ability of subjects to finish with object recognition and eye-hand coordination tasks.3)Aiming at the small visual field problem of current retinal prostheses in clinical application,by using image processing algorithms,the image information of the wide visual field can be compressed into a small visual field to expand the perceived visual field of prosthesis recipients.Although image compressing or retargeting algorithms have been used for expanding the perceived visual field,these algorithms lead to either the decrease of visual acuity or the distortion of foreground information in scenes.To deal with these deficiencies,in the thesis,we proposed an optimized content-aware image retargeting method based on the proposed global salient object detection algorithm to expand the perceptible visual field of retinal prosthesis recipients while maintaining visual acuity as much as possible.We evaluated the proposed method on two psychophysical experiments based on object detection and object recognition under stimulated prosthetic vision.The results demonstrated our method significantly improved the ability of subjects to detect and recognize objects under low-resolution prosthetic vision.The research objective of this thesis is to improve prosthesis recipients' vision perception,enhancing their ability to complete visual tasks,and contributing to the design of information processing modules for retinal prostheses by research on the information optimization processing and cognitive mechanism under prosthetic vision.At the same time,it will also provide scientific experimental basis for the cognitive mechanism of visual function restoration.