Optimization Algorithm For Irregular Artificial Vision Based On Transformer Saliency Detection

As a severe disease,blindness has a significant impact on human life,especially age-related macular degeneration and retinal pigment degeneration,which are currently incurable.Retinal implants have been proven to be an effective method for restoring visual function.However,due to technological,material,and biological safety limitations,only a limited number of microelectrodes can be implanted.Since the last century,many research teams at home and abroad have been developing a high-density visual prosthesis.However,they have not yet entered substantial clinical stages.The number of microelectrodes that can be implanted in existing visual prostheses is still limited.Additionally,the vision generated by stimulating the implanted microelectrodes is usually of very low gray level.Implant recipients also report irregular artificial vision generated by visual prostheses,with drop-out and distorted phosphenes,which make it difficult to complete some visual tasks in daily life.Therefore,implant recipients cannot use significant features such as color and brightness contrast to complete daily tasks at low resolution.Many functional visual tasks such as target detection and target recognition are included in daily life,among which the most widely used is object recognition.Based on this,this study proposes an optimal representation strategy of visual information for object recognition and correction of irregular array properties for indoor scenes based on simulated retinal prosthesis vision in order to improve the visual function of implanters.The content and work conducted in this study include the following:1.Study the saliency detection methods in the field of computer vision,select the Transformer model-based saliency detection algorithm(Visual saliency transformer,VST)that can meet the visual prosthesis application scenario in this study as a base method for this research work,and choose to use the nearest neighbor search algorithm as a method for irregular array correction and the final simulation experiments of the dynamics are carried out.The edge optimization is added to the nearest neighbor search,and the edge concavity is calculated to filter the most suitable phosphenes in the set of nearest neighbor search results.2.Designing object recognition task in indoor scenes,for object recognition problem under artificial vision conditions,using Transformer based saliency detection method to extract salient objects in real time and accurately,several image processing optimization expression strategies are proposed.These strategies are all for object recognition in indoor scenes,which are Edge mask-based pixelation(EMP),Saliency mask-based pixelation(SMP),Edge mask-based pixelation and Irregular array correction(EMP-IC),and Saliency mask-based pixelation and Irregular array correction(SMP-IC).3.The simulation experiments carried out in this study use portable head-mounted display devices,miniature wired cameras,portable 5GWifi and portable laptops to build the retinal prosthesis vision simulation experimental platform and construct the system required for the experiments.Real-time experiments on object recognition in indoor scenes are conducted on the built experimental platform and experimental system.Among the experiments,Direct low pixelization(DP)was used as the control group,and other processing methods were compared with Direct low pixelization as the artificial vision optimization expression strategy.Eighteen subjects were tested under simulated retinal prosthesis vision conditions,and head movement in degrees(HMID),task response time and the number of successful small target objects used during the subjects’ experiments were recorded.For data analysis,ANOVA one-way ANOVA and least significant difference method LSD were used to analyze the significant differences between EMP,SMP,EMP-IC,SMP-IC strategies and DP strategies.The experimental results demonstrate that several image processing strategies proposed in this study help to improve object recognition and task performance in indoor scenes.The results proved that the subjects’ task performance under the EMP-IC and SMP-IC strategies proposed in this study was the best both in terms of subjects’ average HMID,task response time,individual object recognition response time,object recognition accuracy and number of successful small target recognition.The average HMID of subjects under the EMP-IC strategy was 64.27 ± 15.59 deg,and the average HMID under the SMP-IC strategy was63.39 ± 15.38 deg.Under the EMP-IC strategy,the response time taken by subjects to complete the task was 26.10 ± 6.33 seconds;the average accuracy rate of object recognition was 98.03% ±23.78%.Under the SMP-IC strategy,the average response time used by subjects to complete the task was 22.28 ± 5.40 s and the average recognition accuracy was 94.31% ± 22.87%.This study proposes a strategy for optimal representation of visual prosthesis images based on Transformer saliency detection and irregular array correction,which meets the real-time requirements in experiments while maintaining accurate extraction results,and proves the feasibility of the strategy through retinal prosthesis visual simulation experiments,which improves the visual task completion ability of implanters in extracting object information in object recognition tasks in indoor scenes,helps them extract effective visual information to better complete object recognition tasks in indoor scenes,and provides a reference for future studies on retinal prosthesis optimization.