Sentiment Analysis Of Visual-Textual Comments Based On Multi-Level Attention

With the continuous popularization and promotion of social networks,compared with traditional text description,people were more inclined to post illustrated content to express their feelings and opinions.However,the sentiment features of images are complex and subjective.The existing visual-textual sentiment analysis methods only considered the high-level semantic relation between images and texts,but paid less attention to the correlation between the low-level visual features and middle-level aesthetic characteristics of images and the sentiment of texts.Therefore,focuses on introducing hierarchical image information to enhance the effect of visual-textual sentiment analysis.The specific work package is the following two parts:In order to verify the effectiveness of multi-level image features in the visual-textual sentiment analysis task,an image sentiment analysis method based on multi-stage feature fusion was proposed.In this method,multi-branch convolutional neural network is used to extract visual features of different levels,and three fusion methods of multi-stage fusion(MSF),deep supervised fusion(DSF)and multi-stage attention fusion(MSAF)are proposed to mine the sentiment co-occurrence between images and texts.The experimental results from the public sentiment datasets show that the multi-stage features fusion method,which comprehensively analyzes the hierarchical sentimental connections between text features and image features,can effectively improve the effect of visual-textual sentiment analysis.Since spatial attention has achieved remarkable results from various kinds of prediction tasks,a new method of visual-textual sentiment analysis based on multi-level spatial attention(MLSA)is proposed.The method introduces the spatial attention mechanism into the framework of hierarchical feature fusion,and build a multi-layers structure of spatial attention module.The convolutional neural network features of different levels are weighted by spatial attention which lead by the textual feature,and weighted feature is used as the convolution input of the next layer until the last convolution result is obtained.Compared with other methods,the experimental results show that the method based on multi-level spatial attention can effectively enhance the ability to capture the emotional semantics of text and image,so as predicting the overall sentiment of visual-textual comments more accurately.