| With the development of the Internet,social media has become an important platform for users to share information and express emotions.Sentiment analysis,as an important research topic in social media analysis,serves as the foundation for many practical issues such as public opinion monitoring,word-of-mouth marketing,and product recommendations,and it holds significant research significance and application value in fields like public safety and commerce.With the popularization of multimodal social media platforms for online comments,the form of online comment data has shifted from text to multimodal formats such as text and images,enhancing the efficiency of user information acquisition.However,due to the complexity and heterogeneity of multimodal data,it becomes more challenging for computers to comprehend multimodal data content.In response to the insufficiency of text semantic extraction in existing sentiment analysis models,the inadequate capability to handle multi-classification problems,and the insufficient exploration of common emotion-related features in multimodal data,this study conducted the following research work:Firstly,for the binary sentiment classification problem,we proposed the Ro BERTaBiLSTM-Attention model.This model addresses the low classification accuracy of traditional machine learning methods by adopting a structure that combines an upstream pre-training model and a downstream network model.It utilizes Ro BERTa to obtain text vectors,which are then processed by the BiLSTM-Attention network that can handle long-range dependency problems and focus on important parts of the input sequence.Finally,the softmax function is applied to obtain the classification results.Experimental comparisons have demonstrated the superiority of this model.Secondly,in the context of text sentiment multi-classification tasks,an improvement is made to the structure of the Ro BERTa-BiLSTM-Attention model,leading to the proposed Ro BERTa-MEN model.This model utilizes the word embeddings generated by Ro BERTa,which are then processed through a BiLSTM-Multi Head Attention mechanism to obtain a new sentence embedding.Subsequently,this sentence embedding is concatenated with the [CLS] sentence embeddings from the last three layers,which have been subjected to BiLSTM-Attention processing,resulting in a feature vector.This concatenation is then inputted into a fully connected layer to obtain the classification results.Extensive experiments conducted on the dataset have provided substantial evidence of the effectiveness of the two modifications applied to the embedding information.Moreover,based on the research of single-modal text sentiment analysis,this study proposed a Ro BERTa-based contrastive learning model for multimodal sentiment classification involving text and images.This model simultaneously considers the feature learning of each modality and the fusion between modalities.It innovatively uses an improved Ro BERTaBiLSTM-Attention model to process text information and efficiently extracts image information using Res Net50.Multiple layers of Transformer-encoder modules are employed to align and fuse the token-level features of text and images.In addition to the sentiment analysis task,we also designed label-based contrastive learning tasks and data-based contrastive learning tasks to assist the model in learning common emotion-related features in multimodal data.Extensive experiments on the multimodal dataset have demonstrated that compared to the baseline model,the Ro BERTa-based contrastive learning model proposed in this study achieves higher classification accuracy in multimodal sentiment classification. |
