Key Technologies For Fine-grained Sentiment Analysis Towards Multimodal Data

With the approach of mobile-Internet era,social media becomes world wide popular and grows more influential.The study of aspect-based sentiment analysis(ABSA)on social media has become a hot topic recently.However,there are obvious differences between the form of social media and traditional text.On the one hand,because of the informal nature of social media,there might be abbreviations,grammar mistakes and typos within the text.On the other hand,the corpus of social media always contain not only text but images.On the basis of aforementioned characteristics,the research of ABSA on social media has high academic and practical value.The thesis mainly focuses on two sub-tasks of ABSA: 1)Aspect Term Extraction(ATE)on multimodal data of social media.The objective of the task is to extract every single solid aspect term within a given text-image pair;2)Aspect Term Polarity(ATP)on multimodal data of social media.The task aims to judge the polarity of aspect terms within multimodal data.Specifically,the main studies of the thesis are of the following parts:Firstly,the thesis purposes a method for multimodal ATE based on Region-aware Alignment Network(RAN).The model first introduces BERT model to capture semantic information of context.Then the model leveraged character-level embedding of words targets on the out-ofvocabulary problem.The model uses object feature extracted by object detection network FasterRCNN as image feature.Next,the model builds a co-attention network to interact text features with image features and fusion multimodal features.Finally,the model uses a filtration gate to filter out noises of the related image.Secondly,the thesis brings out Grammar-Oriented network based on PREtrained Models(GOPREM)to solve the task in and end-to-end fashion.This model uses LXMERT to model fusioned multimodal features,then decide the extent of the fusion by the unique multimodal output by LXMERT.Moreover,the thesis leveraged Grammar attention layer to strengthen the grammar interpretability.In addition,the model use CRF model to decode aspect term and its sentimental polarity simultaneously.At last,the thesis does a wide range of experiments on purposed models.The result of experiments shows that both models perform well on their targeted tasks.In particular,RAN and GOPREM enhances 1.8% and 2.6% of F1-measure compares to their best baseline model,respectively.The thesis does ablation study on both models and prove effectiveness of each unit within.Furthur,the thesis analyses weekness of the models for future work.