Research On Full Reference Light Field Image Quality Assessment Based On Vision Clues Learning

Vision is the most direct and profound perception of the world.With the development of science and technology,the demand of people for applications such as virtual reality and augmented reality is gradually increasing.The light field technology is an emerging technology to achieve realistic and immersive visual experience.Light field can better provide people with scene perception and environment interaction,and its development prospect is broad.However,in the process of acquiring,encoding,compressing and transmitting light field images,how to ensure the quality of light field images has always been a challenging problem in the application of light field images.Based on the characteristics of human visual perception,establishing a light field image quality evaluation system has obvious academic significance and application value.In this paper,combining visual cues,machine learning and deep learning,three light field image quality evaluation models are proposed.The main work and contributions are as follows:(1)Aiming at the problem of how to artificially and accurately simulate human eyes to extract rich visual cues and to fuse visual cues,a light field image quality assessment model based on feature fusion of rich visual cues is proposed.Based on the perception characteristics of the human visual system to real scenes,combined with multi-scale visual perception characteristics,visual cues such as lightness masking and contrast masking: extract local brightness and geometric structure features;combine singular values to reflect image hierarchical structure characteristics to obtain clarity and intrinsic structural features;Combined with the parallax information,the gradient cross-mapping is used to obtain the depth texture structure features.The above features were regressed by support vector machine based on genetic algorithm to obtain the final quality score.Experiments show that the proposed model has better performance than the existing light field image quality assessment models.(2)Aiming at the cumbersome manual extraction of visual clue features and the difficult feature level of the full-reference quality assessment method,a light field image quality assessment model based on Siamese networks is proposed.Considering the different perceptual characteristics and fusion mechanisms of the human visual system for visual signals of different frequencies,the light field sub-aperture image is filtered by a contrast-sensitive function and synthesized into a central eye diagram;the central eye diagram of the reference image and the distorted image is sent to the weight sharing in the Siamese network,the distance in the feature space is obtained after feature extraction,and the light field image quality score is obtained through pooling regression.Experiments show that the image quality assessment performance of the model is similar to the subjective quality judgment ability of human eyes.(3)Aiming at the problems that the light field dataset is small,the light field image has high dimensionality,contains a large amount of visual clue information,and the global visual clue characteristics of light field quality are difficult to learn,a Vi T light field image quality assessment model based on an adaptive attention mechanism is proposed.First,the light field sub-aperture image sequence is sent to the pre-trained Res Net50 model to extract the local structural features of the light field;then for the limitations of the small light field dataset and the difficulty of learning the global visual clue features of the convolutional neural network,a new method is designed.Vision Transformer network with adaptive attention mechanism to obtain accurate light field image quality scores.Experiments show that the model can better learn the spatial structure of the light field on a small dataset of light field images.The multi-view subaperture image input can enable the model to learn the features of the light field angle domain that were previously ignored,which also can fit perceptual characteristics of the human visual system to achieve accurate light field image quality assessment.