Research On Depression Tendency Detection Method Based On Text Pre-Trained Model

Depression causes great harm to human physical and mental health and even endangers society.Therefore,it is essential to find the early symptoms of depression and treat them in time.An earlier state of people with depression is called depressive tendency.Like most normal people,people with depressive tendency share their stories,express their emotions,and seek help and support on social media platforms.Therefore,the massive social platform data allows us to mine the characteristics of depression tendency and discover the patients with depression tendency.However,how to effectively utilize social media platform data to find important features to identify users’ depressive tendency has become a challenging problem.Although there are many text-based methods for detecting depression tendency,the detection results are not satisfactory.First,the feature representation of the text data published by the user is not rich enough,the multimodal features of the text are not fully utilized,resulting in the lack of semantics information of the text;Second,some methods do not attach sufficient attention to the words or sentences that play a vital role in the detection of depression tendency,and do not focus on learning their features;Finally,most of the current text mining methods do not fully mine the syntactic structure features of the text.They cannot solve the semantic ambiguity of words in the text on social platforms,which seriously reduces the effect of detecting depression tendency.Pointing to the above problems,this thesis studies the characteristics of social media texts,text-based mining methods,and depression tendency detection methods.It proposes a series of depression tendency detection methods based on text pre-training models.The main contributions of this thesis are as follows:(1)Propose a multimodal feature and text pre-training based method for depressive tendency detection(MTDD)to solve the problem of insufficient feature representation of text data and lack of text semantic information,resulting in poor detection of depression tendency.First,the MTDD model is a hybrid model based on a deep neural network,combined with CNN(Convolutional Neural Network)and Bi LSTM(Bidirectional Long Short-Term Memory)network,to avoid the problem of the poor generalization ability of a single model for depression tendency identification;Second,the MTDD model performs vector representation learning based on multimodal features of text,including text features,semantic features,and domain knowledge,making the model more robust.(2)Present a depression tendency detection method based on embedding from language models,hierarchical attention networks,and text pre-training(E-HAN)to solve the problem of insufficient attention to keywords or sentences and no focus on learning their features.First,use the pre-training model and Embedding from Language Models(ELMO)to obtain word embeddings,and fuse the emotional features and part-of-speech features of words to form multi-granularity features of words,and obtain rich word feature representations;Second,feature extraction is performed separately from the word-level and sentence-level.An attention mechanism is introduced to capture the features of words and sentences and give them different weights to highlight important feature information and improve the detection performance of the model.(3)Propose a depression tendency detection method based on text pre-training and dependency parsing(PMDT)to solve the problems of semantic ambiguity of words and insufficient syntactic structure features.First,the idea of segmentation is proposed.The lengthy text is divided into several short texts,and the word embedding vector of each short text is weighted and averaged to obtain the feature vector representation of the entire long text;Second,the binary dependencies between the components in the sentence are obtained according to the results of the dependency syntax analysis to construct the dependency matrix.The feature representation of each word is constructed using syntactic information,and the relational features between words are learned through Graph Convolution Network(GCN);Third,the BERT(Bidirectional Encoder Representation from Transformers)word embedding feature and the sentence grammatical structure feature are combined as the feature representation of the text,combined with the Bidirectional Recurrent Neural Network(Bi RNN)model to extract features and output prediction results.We conduct extensive experiments on multiple public datasets and use various metrics to evaluate the predictive performance of our works.The results show that the three methods proposed in this thesis work better than the state-of-the-art text-based depression tendency detection approaches.