No-Reference Image Quality Assessment For Screen Content Images

With the rapid development of interactive multimedia technology,screen content images are widely used in multimedia applications such as screen sharing and online education.However,various distortions will be introduced in the process of screen content being generated and displayed on its terminal,which will lead to the degradation of image quality and greatly affect the user’s visual experience.Therefore,it is crucial to design an objective image quality assessment method to evaluate the quality of screen content image.To this end,this paper explores and studies the image quality assessment method of noreference screen content image.The main contents are as follows:Considering the visual difference between the text content and the image content of the screen content image,a no-reference image quality assessment method based on regional feature fusion is proposed.First,the screen content image is divided into image area and text area.For the structural information contained in the text region,the multiorder derivative direction histogram is used to characterize it.Meanwhile,the text region contains large text content,and we use the sharpness metric of the document image as a visual quality feature.Considering that the image area contains rich texture information,we use the local binary pattern histogram on the gradient map to extract the texture information of the image,and extract the statistical features from the normalized luminance map,and then calculate the energy of the wavelet subband as the sharpness.Finally,the extracted quality features are fused and the support vector machine is used to map the quality features into subjective scores.The experimental results show that,compared with the existing assessment methods,the proposed algorithm is highly consistent with the subjective evaluation results.Existing deep learning-based quality assessment models only learn features from grayscale images and ignore the color information of images.At the same time,the deep learning network of a single image stream cannot fully extract the features that reflect the image quality.Inspired by the two-stream convolutional neural network,we propose an improved two-stream convolutional neural network by introducing a transfer learning approach and using the pre-trained residual network as the feature extraction network.The color,intensity and structural features of the screen content image are extracted from the RGB image and the gradient image respectively through the two-stream network,and the abnormal color or saturation caused by the distortion of the screen content image and the changes in the high-frequency area of the image are fully captured.In the databases SIQAD and SCID,the Pearson Linear Correlation Coefficient(PLCC)of the proposed algorithm is improved by 3.7% and 4.1%,respectively,compared with the state-of-the-art algorithms in the comparison.