A Research Of Sentiment Classification Algorithm And Application Based On Attention Mechanism

Sentiment classification refers to the analysis and judgment of the sentiment polarity of text,and is applied to opinion mining,sentiment recognition,public opinion analysis and other aspects.Attention mechanism is widely used in the field of natural language processing,and has a high accuracy rate on many classification tasks.Both recurrent neural network and attention mechanism are end-to-end structures that have the ability to combine context.Recurrent neural networks learn along time and can remember se-quential information.But when the sentence is too long,even the variant LSTM of RNN cannot learn the distant word information.Self-attention mechanism can be used as an encoder,so that each word can obtain global context information.For sentiment classifi-cation algorithms,it is important to embed sentiment information into the network for rich text representation.Although self-attention mechanism can pay attention globally,it also introduces noise words from it.The more complex context is enough to confuse the audio-visual of self-attention mechanism,and there are situations where each word with emotion is important,so there is still much room for improvement of self-attention mechanism.Considering the integration of part-of-speech into self-attention mechanism,this pa-per proposes an sentiment classification algorithm based on self-attention mechanism of fused part-of-speech embedding.It specifically includes two sentiment classification al-gorithms,which are the sentiment classification algorithm based on Pos-IdSA(Part Of Speech Independent Self Attention)and the sentiment classification algorithm based on Pos-ItSA(Part Of Speech interactive Self Attention),and perform parameter optimization experiments and algorithm performance analysis experiments on SST-2 and MR datasets.The experimental results prove that the accuracy of sentiment classification algorithm with fusion of parts of speech is higher than the baseline algorithm.It shows that after adding part-of-speech embeddings,the network can learn the grammatical relationship between different parts of speech.The fused part-of-speech features help improve the accuracy of sentiment classification algorithms.In the face of long and complex contexts,the global attention mode of self-attention mechanism will be disturbed by noise words.This paper proposes a sentiment classifi-cation algorithm based on the band width self-attention mechanism,which specifically includes two sentiment classification algorithms,namely the sentiment classification al-gorithm based on RTA-WSA and the sentiment' classification algorithm based on WSA-RNN.Based on Glove's experiments,comparing the expressiveness of the three attention mechanisms on the SST-2 data set,the effectiveness of the WSA structure is proved.Based on the experiments of RNN,the connection modes o f RNN and Attention are divided into series and parallel.RTA-WSA mainly parallels self-attention mechanism with the RNN,and proposes a multi-width self-attention mechanism to extract multi-angle features with different attention ranges.WSA-RNN uses the band width self-attention mechanism be-fore RNN structure,which supplements the long-term word information for RNN in ad-vance,and makes up for the shortcomings of RNN.The effectiveness of the sentiment classifi cation algorithm based on the band width self-attention mechanism is verified by the algorithm performance analysis experiments,and it is proved that the band width self-attention mechanism is helpful to improve the accuracy of the sentiment classification algorithm.