Blind Image Quality Assessment Method Based On Multi-stream Networks And Meta-learning Model

With the widespread application of multimedia technology,the importance of image quality assessment in image-based applications is growing rapidly.Its purpose is to establish an efficient and accurate model to replace the human eye in accurately evaluating image quality.For most real-life application scenarios,the distorted images that need to be evaluated cannot obtain their reference images.Therefore,this paper focuses on no-reference image quality assessment,and the main research content includes:(1)Considering that reference images play an important role in improving assessment performance in the field of image quality assessment,a generative adversarial network is improved to generate high-quality pseudo-reference images,and a novel three-stream quality assessment network structure(TSAIQA)is proposed,where the inputs of the three streams are distorted images,pseudo-reference images and gradient feature maps of distorted images,respectively.used to extract rich quality-related features.The distorted image stream focuses on the feature information after distortion,the pseudo-reference image stream is used to supplement the feature information lost due to distortion,and the gradient stream explicitly extracts quality-related structural features.Improved spatial and channel attention mechanisms are embedded in the distorted and pseudo-reference image streams to extract features that match the attention of the human eye.The algorithm experiments on four classical image quality assessment datasets and two recent large-scale datasets,and the experimental results prove the effectiveness of the proposed network model and the improved attention mechanism.(2)The image quality assessment model obtained by the above algorithm through the design of a fine-grained network structure has shown a good performance.However,considering that deep learning requires a large number of data labels during the training process,and the limited size of the existing image quality assessment dataset,this may lead to overfitting during the network training process.To avoid this situation,the concept of meta-learning is introduced,and a blind image quality assessment method based on multi-resolution spatial-frequency domains characteristics and meta-learning is proposed.First,during the construction of the meta-training set,ten sub-task sets related to quality assessment are obtained by extracting the feature maps of distorted images and their resolution-processed difference maps in the spatial-frequency domains from the training image dataset,which are used to learn a better hyperparameter to the model and obtain a priori model of quality assessment.Next,the target set of tasks to be evaluated is fed into this priori model of quality assessment,which is meta-tested to obtain the final image quality assessment meta model.The model combines meta-learning methods and the multi-resolution spatial-frequency characteristics of images.It adopts a two-level gradient optimization training approach and can achieve autonomous parameter tuning.The model performs well on four synthetic distortion datasets and two real distortion datasets,especially when evaluated across datasets,it demonstrates superior performance.