Research On Image Quality Assessment Technology Based On Visual Perception

Digital images are an important part of our daily lives,as our society becomes highly digital,and there is an on-going demand for high-quality images.However,digital images typically suffer from distortion of different types and degrees that are introduced during image acquisition,transmission,compression,storage,and other image-processing procedures.To ensure that the quality of an image satisfies the user’s requirements,image quality assessment has been integrated into many image-processing systems.With the rapid development of multimedia technology,stereoscopic images can provide immersive experiences and vivid visual effects to the viewers.Compared to a two-dimensional(2D)image,a stereoscopic image involves depth information and consists of two views.Therefore,stereoscopic image quality assessment is more complicated than 2D image quality assessment.Stereoscopic image quality assessment includes quality assessment of the left and right views,binocular rivalry,visual fatigue,visual discomfort,and image depth perception.Based on the perceptual characteristics of the human visual system,this paper conducts research on the quality assessment of 2D images,singly-distorted stereoscopic images,and multiply-distorted stereoscopic images to further improve the understanding of distortion and improve the consistency of objective and subjective quality assessments.Specifically including:Firstly,a full-reference image quality assessment method based on pairwise learning to rank is proposed.Objective assessment is less capable of distinguishing the small quality differences between images than subjective assessment.Because the pairwise comparison is a natural and effective way to obtain subjective image quality scores,this paper proposes to compare image quality in pairs using pairwise learning to rank.First,a large number of image pairs are composed,and features are extracted for each pair of images and their preference labels are calculated as training labels.Then a pairwise preference model is obtained by training a binary classifier using the features and labels.Because image quality is affected by the masking effect,this paper proposes extracting frequency-aware quality features by adapting the state-of-the-art image quality assessment metrics.The learned pairwise preference model is then used to predict the preference between pairs of images in the testing dataset.The quality of each image is finally computed as the number of preferences.Experimental results on four image quality assessment databases validate that the proposed pairwise learning to rank-based image quality assessment method achieves higher consistency with human subjective assessment than the state-of-the-art image quality assessment metrics.Secondly,a no-reference singly-distorted stereoscopic image quality assessment using image registration and multi-task learning is presented.Scene discrepancy between the left and right views presents more challenges to image quality assessment for stereoscopic images than that for monocular images.Existing no-reference stereoscopic image quality assessment metrics cannot achieve good performance on asymmetrically distorted stereoscopic images.This paper addresses scene discrepancy by image registration and computes the registered distortion representation based on the left and registered right views to represent the distortion in the stereoscopic image.Because different distortion types influence image quality differently,a multi-task convolutional neural network is employed to learn image quality prediction and distortion-type identification simultaneously.A one-column multi-task convolutional neural network model that learns from the registered distortion representation is first presented.Then,the one-column model is extended to a three-column model,which also learns from the left and right views.Experimental results on the LIVE 3D and NBU 3D IQA databases validate the effectiveness of the proposed registered distortion representation and multi-task convolutional neural network architecture.The proposed one-column and three-column models outperform other state-of-the-art stereoscopic image quality assessment metrics,especially for asymmetrically distorted stereoscopic images.Thirdly,a no-reference multiply-distorted stereoscopic image quality assessment based on binocular rivalry is presented.It is more practical to investigate multiply-distorted stereoscopic images,because images are prone to various distortion types in capturing and processing stages.The mixture of multiple distortions leads to complex binocular visual behavior of multiplydistorted stereoscopic images,so the existing no-reference singly-distorted stereoscopic image quality assessment methods cannot obtain satisfactory results on multiply-distorted stereoscopic images.Because binocular rivalry influences the final stereo images greatly,in this paper,the result of binocular rivalry is presented by merging left and right views into a cyclopean image.Considering that the color and intensity of the pixel in the RGB image can well reflect the information of the distorted image,in the paper,a grayscale cyclopean image is further converted to an RGB cyclopean image through tone mapping.Finally,a multiply-distorted stereoscopic image quality assessment method based on a double-stream convolutional neural network is proposed.The two subnetworks are used to extract quality features from the color cyclopean images and registered distortion representation,respectively.Experimental results demonstrate that the performance of our proposed model is significantly improved compared to the existing models on the multiply-distorted stereoscopic image databases.In conclusion,this paper investigates to establish image quality assessment methods that consistent with visual perception on human visual system,combining the masking effect,scene discrepancy of binocular stereopisis,binocular rivalry characteristics,and multi-task neural networks and double-stream neural networks.The proposed model can obtain more descriptive perception features to describe image distortion,which further improves the performance of the image quality assessment method.