Blind Image Quality Assessment By Combining Attention Mechanisms And Multi-level Features

Digital images are affected by different factors in every step of acquisition,processing,coding,transmission and storage,resulting in different types of distortion.These distortions will affect people’s understanding and feeling of image content.In order to evaluate the image quality of different distortion types effectively,a general no reference image quality assessment method based on joint attention mechanisms and multi-level feature fusion is proposed in this thesis.The main achievements are as follows:(1)For general types of distortion,this thesis proposes a saliency enhanced two-stream convolutional network(SETNet)for no-reference image quality assessment.The proposed SETNet contains two subnetworks of image stream and saliency stream.The image stream focuses on the whole image content,while saliency stream explicitly guides the network to learn spatial salient features that are more attractive to human.Then the spatial attention module is employed in saliency stream to refine salient features,and the refined salient features are used as weights to guide the image stream features enhancement.Meanwhile image stream and saliency stream features fusion strategy is proposed to integrate features at corresponding layer,and the integrated features are taken as an input of the next layer feature fusion.Finally,the integrated features of each layer are further refined with the proposed dilated convolution based channel attention(DCA)module to optimize features in channel level,and the final quality scores are predicted by using multiple levels features and weighting strategy.The experimental results of the proposed method and several representative methods on four synthetic distortion datasets and two real distortion datasets show that our SETNet has higher prediction accuracy and generalization ability.(2)Based on the above methods,this thesis proposes a no reference image quality assessment method which is based on multi-task recovery image and multi-stream feature fusion(GMSNet).Firstly,we train the recovery network by putting the distorted image into the multi-task recovery network in order to obtain a reliable fake reference image.The multi-task network includes two sub-networks,a fake reference image prediction network and a saliency image prediction network,both of them share convolution neural network feature extraction layer.Secondly,the fake reference image,the distorted image,the structural difference image of the fake reference image and the distorted image,the saliency image corresponding to the distorted image are putted into the multi-stream quality assessment network.At the same time,the saliency image is used to guide the features of the other streams in this process,to improve the performance of the algorithm.Finally,the quality is evaluated by fusing the features of these four streams.The proposed algorithm shows good prediction performance and generalization ability on four synthetic distortion datasets and two real distortion datasets.Compared with the first algorithm proposed in this thesis,it has better evaluation performance.