Image Quality Assessment For Human Visual Perception Characteristics

Visual information is the main digital information resource in the era of big data,and images and videos are the main sources for humans to obtain visual information from the outside world.The human visual system is the receiving and sensing terminal of visual information,which is an efficient and complex information perception processing system.Therefore,how to quickly and accurately obtain information from the massive data in line with human visual perception has become a hot and difficult research topic.It means how to effectively improve the computer's ability to process visual information and evaluate the quality of images and videos based on the human visual perception.In a word,the image quality evaluation for human visual perception characteristics can be regarded as the brain guiding the extraction and processing of the underlying visual information through the visual system to analyze and evaluate the visual perception of image/video clarity,data volume or depth perception,which is the basis for guiding a series of issues such as visual information collection,coding,compression,reconstruction,processing and display.Starting from the characteristics of human visual perception,this dissertation focuses on the visual acuity of images,the comprehensive perception of image quality,the stereoscopic depth perception of 3D videos based on the binocular vision system,and the corresponding modeling applications in digital image processing.In the process of visual perception,human eyes cannot distinguish visual content changes under a certain threshold due to the limited ability of human vision.The just noticeable difference threshold represents this visual perception and distinguishing ability,which can effectively remove the image redundant information,and improve the image compression and coding performance.In the late stage of visual perception,image quality determines the sufficiency and accuracy of human visual perception for image content,while image super-resolution is dedicated to the reconstruction and restoration of image content.Therefore,the objective quality assessment of super-resolution images can predict the reconstruction performance of super-resolution algorithms.Moreover,it can be used to evaluate and optimize the performance of superresolution reconstruction algorithms.Since the human eyes have stereoscopic depth perception characteristics,the depth quality evaluation of 3D video reflects the binocular vision system's perception of stereoscopic depth information,which can be used in processing systems such as 3D video compression and encoding.In short,there are following three main research works: just noticeable difference threshold estimation,super-resolution image quality assessment and 3D video depth quality evaluation.The specific research content and contributions are as follows:(1)For the visual sensitivity of human visual perception,this article is focus on the orientation selective characteristics of human visual,and proposes a feature extraction operator for structural complexity of visual content;According to the subjective visual sensitivity difference of different structure regions,the mask effect of these regions is estimated;Combining the contrast sensitivity of the DCT coefficient frequency and the brightness adaptability,a novel DCT-based JND model is proposed.Compared with other DCT-based JND models,the proposed models in this research have two innovative breakthroughs.First,the proposed mask model based on the orientation complexity is more accurate in estimating the mask effect of different visual content than the previous contrast mask,which is more in line with the subjective visual perception of the human eyes.Then the whole process of JND estimation is completely based on the DCT coefficient,without cross-pixel domain operations,and can be directly applied to the image compression and coding system.While improving the estimation accuracy of the JND algorithm,it also ensures the practicability.(2)For the comprehensive experience of image quality in the later stage of human vision,this dissertation focuses on the shortcomings of image super-resolution performance evaluation,such as peak signal-to-noise ratio PSNR and structural similarity SSIM,which are more and more difficult to meet performance comparisons of the existing deep learningbased super-resolution algorithms in terms of consistency with subjective visual perception.This article first organized the subjective evaluation experiment of super-resolution image quality,constructed a reliable super-resolution image quality evaluation database,and verified the shortcomings of existing objective quality evaluation indicators for superresolution images.Then,based on the free energy theory and the internal generation mechanism of the brain,a visual content prediction model is proposed to predict the texture and structural characteristics of the visual content of different regions.Finally,by calculating the similarity between the visual content prediction model of the super-resolution image and the reference image,an objective quality evaluation index specially used for comparing the performance of super-resolution algorithm is proposed,which improves the consistency with subjective visual perception and assist the design and optimization of the super-resolution algorithms.(3)For the evaluation of stereoscopic depth quality based on binocular vision perception characteristics,3D video depth perception may be affected by external noise factors such as encoding,compression and so on.This article organized a series of subjective depth perception experiments to explore the relationship between depth perception and basic visual elements such as frequency,orientation and movement.Then,combined with the depth perception cues and the spatiotemporal structural characteristics,the binocular,monocular and motion depth features of 3D videos are extracted respectively for modeling;Based on the 3D video depth perception evaluation database,these depth features are trained by support vector machine to obtain a 3D video depth quality evaluation model.The main contribution of this research is to build a database for quality evaluation of 3D video depth perception;propose a framework to estimate the 3D video depth quality by the depth perception features from three aspects of binocular,monocular and motion;achieve a reference-free evaluation of the 3D video depth quality.The proposed depth quality evaluation model is superior to the existing depth quality indicators in the terms of subjective depth perception consistency,and is more in line with the depth perception of the human visual system.