Image Quality Assessment Based On Deformable Convolutional Neural Networks With Gradient Fusion And Bilinear Attention Mechanism

Deep learning is an emerging research direction in the study of image quality assessment.This technology has shown great power in the processing of multimedia audio-visual materials and medical images as well as in some other fields.Therefore,it is of great importance to study and improve the deep-learning algorithms in no-reference image quality assessment both theoretically and practically.After investigating the research background,significance and main research methods of this topic,the gap in conventional prediction methods was discovered.These methods mainly include traditional digital image detection,wavelet decomposition algorithms,support vector machines,artificial neural network(ANN)methods,etc.,which recognize image quality assessment as the task of fractional regression prediction.This may work well in predicting images with extremely low or high distortion,while fail in the prediction of some moderately distorted images due to the obvious shortage of fine-grained feature extraction.Aiming to circumvent this constraint,this thesis proposes two improved models,which are the improved CNN quality assessment model with fusion gradient information,and the bilinear attention-gated deformable CNN image quality assessment model.The model combines local RGB information and local gradient information to enhance the sensitivity of local edge blurring in images.Meanwhile,using bilinear attention mechanism can help the model extract second-order features and use the attention in attention(AIA)mechanism to extract the relationship between second-order local features and global features.In addition,by using the deformable convolution,which can adaptively adjust the size of the convolution kernel at different levels,the model can effectively capture different scale information in the image.Finally,the model adaptively learns each feature through the gating mechanism,adjusts its weight,and then evaluates the image quality more accurately.The experiments were carried out with two data sets,namely,TID2013 and LIVE datasets.For the improved gradient information fusion CNN quality evaluation model,on the TID2013 dataset,the PLCC index improved from 0.895 to 0.910,with an increase of 1.5%;the SROCC index increased from 0.871 to 0.883,increasing by 1.2%,and the KROCC grew from 0.863 to 0.876,with an increase of 1.3%.For the improved gradient fusion-based bilinear deformable convolutional neural network,the PLCC index went up from 0.910 to 0.923,by 1.3%;the SROCC index rose from 0.883 to 0.904,by 2.1%;and the KROCC improved from 0.876 to 0.901,increasing by 2.5%.It can be seen from the results that the four improvement points proposed in this thesis,i.e.,gradient information fusion,bilinear attention mechanism,deformable convolution module,and gating mechanism,have proved their ability in helping improving image quality assessment on both datasets.It can be concluded that the models proposed in this thesis are effective and have important practical significance in the field of image quality assessment.