No-Reference Image Quality Assessment Based On Weakly Supervised Learning And Data Enhancement

Images may introduce different kinds of distortion during the process of acquisition,compression,transmission and storage.The distortions will cause obstacles to information processing,analysis and expression,and also affect people's understanding of the objective world.Therefore,it is necessary to design a reasonable and reliable image quality assessment algorithm to predict the image quality,so as to more conveniently guide the optimization,improvement and development of the visual information processing system.Image quality assessment has become a hot research issue and has been widely used in computer vision,pattern recognition and artificial intelligence.This thesis designs a deep learning network with visual features,aiming at the difficulties and problems such as complex scenes and insufficient available data in the task of evaluating the quality of natural scenes without reference.Through the methods of weakly supervised learning,feature guided learning and data enhancement,the features of distorted image are well extracted and the no-reference quality assessment model is constructed to obtain the evaluation results which consist with subjective perception.The main research contents are summarized as follows:(1)A visual residual perception optimized network for no-reference image quality assessment is proposed.In the deep learning based quality assessment methods,as the network depth increases,more training data are required.But the amount of images with a reliable subjective score is seriously deficient.Therefore,a divide-and-conquer method is proposed.The method decomposes the image quality assessment into two steps: identifying the distortion degree of the image and extracting the image content features,combining the distortion features to obtain the final visual quality features to regress the image prediction score.Firstly,a distortion recognition network consisting of a shallow convolutional neural network and a long short-term memory network cascade is used to measure image distortion.Under the supervision of weakly labels,the network model can be pretrained with a large amount of training data.Secondly,the deep convolutional neural network is constructed to extract the content features of the image,and the visual residual network is formed with the distortion degree recognition network to improve the feature integrity and return the quality score of the image patches.Finally,through a pooling strategy based on the visual saliency of the image,the evaluation score of the test image is obtained.The experimental results show that the algorithm improves the objective prediction accuracy of distortion significantly,and the model prediction has high consistency with human visual perception.(2)Quality saliency guided nonlocal deep network is proposed for no-reference image quality assessment.At present,no-reference image quality assessment has poor robustness in different natural scenes,the prediction process does not conform to human visual characteristics,and it depends too much on the degree of model training.Therefore,this thesis proposes a deep network model with visual characteristics.Firstly,the visual quality saliency map is obtained on the image under test by combining the human eye perception image quality characteristics with the deep convolutional neural network,the saliency map and the original image are superimposed and synthesized as the input of the model.Secondly,a novel nonlocal deep network model is designed by using the basic structure of the VGG16 and embedding nonlocal modules to measure the long dependence on the space.Finally,according to the characteristics of human vision,an adaptive pooling strategy is used to construct a mapping between the local score and the global score,and the final score of the image to be measured is obtained.The method has achieved good results on four public datasets,and has good robustness against uniform and non-uniform distortion types.It also shows that the model is highly consistent with human subjective perception.Whether it is for analog or real distortion,the algorithm can make more accurate predictions quickly.(3)An enhancement model based on cycle generative adversarial network is proposed for no-reference image quality assessment.The current methods increase the amount of data by slicing images,but the true labels of these image patches cannot be reasonably determined.Therefore,a “dual enhancement” method is adopted in this thesis,that is,the dataset enhancement and image enhancement to achieve no-reference image quality assessment.Firstly,through the cycle generative adversarial network,the source domain to the target domain and the target domain to the source domain generate adversarial learning.Meanwhile,the value of the input features is adjusted in time to achieve a large amount of data.Secondly,through the game of the network model,the quality score distribution of the distorted images in the datasets is learned to solve the lack of training data volume positively,and complete the dataset enhancement.At the same time,the trained generator is used to restore the distorted image to the reference image as much as possible.Finally,the full-reference network is trained by the enhanced large amount of data to obtain the final quality score.This method has achieved excellent performance on four public datasets,and experimental analysis has proved that the dual-enhanced model is effective when few data are available.