Research On Aspect-based Sentiment Analysis Based On Multi-head Attention Mechanism

With the development of Web2.0 and the rise of social networks,more and more users are willing to post online comments about a product or a public event.These comment or review data contain a lot of emotional information and have a great value of data mining.However,review data is often in a big amount,reading and analyzing these data will cost users a lot of time.Sentiment Analysis(SA)is an automatic analysis tool that can help users analyze the sentiment tendencies of other users from the data.The goal of traditional sentiment analysis is to assign sentiment polarity to each document,paragraph,or sentence.This type of analysis can only provide an overall sentiment tendency to users while Aspect-based Sentiment Analysis(ABSA)can analyze the entity or attribute from the sentence in a more fine-grained way.Since it can provide richer information,it has attracted the wide attention of many scholars.This article mainly focuses on the ABSA problem based on deep learning,and attempts to propose efficient solutions to some of the problems in the research.First of all,the inputs of current deep learning method for SA are usually the pretrained word vectors.The existing pre-trained word vectors are trained based on statistical information,but in SA,sentiment information has an important influence on model performance.It can improve the model performance by encoding sentiment information into word representation.Therefore,we propose sentiment word embeddings.Firstly,we use sentiment lexicon,Word Net,and negative sampling technology to construct a corpus of synonyms-nonsynonymous pairs,and then construct a simple three-layer pre-training model to fine-tune(to make a minor adjustment on the existing parameters in order to get a better performance)the existing word embeddings.Sentiment information is encoded into the word vector during the fine-tuning training process of the model,and the word vector after fine-tuning is the sentiment word embeddings we want to obtain.Experimental results show that the sentiment word embedding we proposed can improve the performance of text classification models.Secondly,according to the excellent performance of Transformer,Bert,and other models which are based on attention mechanism in natural language processing,we believe that the attention mechanism has great potential in ABSA,so we propose a deep classification model totally based on multi-head attention mechanism.The attention mechanism can effectively use the associated information between the aspect information and the text to make the model know which entity or aspect is currently analyzed one,and this can help model to perform more accurate classification.The multi-head attention mechanism extends the view of single attention.The mechanism allows the model to focus on more words in the sentence to achieve a better classification performance.The experimental results show that the aspect-based sentiment classification model which is based on the multi-head attention mechanism proposed in this paper has faster run time than state-of-the-art while keeps the classification performance.