Research On Multimodal Depression Recognition Based On Text And Speech Signals

In recent years,people have a variety of psychological pressures due to such as fierce social competition and the COVID-19 epidemic,leading to a continuous increase in the prevalence of depression.Depression is a very serious psychological disease in modern society that affect people’s emotions,behavior and thinking,leading to feelings of depression and despair,as evidenced by persistent depressed mood,sleep problems,changes in appetite,difficulty thinking,self-loathing,and social distancing,etc.At present,the diagnosis of depression depends more on the clinical experience of doctors and the subjective feelings of patients,and lacks objective and accurate identification methods.Therefore,it is of great significance for the diagnosis and treatment of depression to recognize depression tendency quickly and accurately.With the development of artificial intelligence technology,the evaluation of depression tendency of patients through emotional calculation and analysis has gradually become a research hotspot.In this dissertation,text and speech based depression recognition has been studied.The main contents and innovations are as follows:(1)Depression recognition based on speech.In this dissertation,the DecisionTree,KNN,SVM,LSTM and Bi LSTM models are used to conduct emotional analysis on speech data.By comparison,the Bi LSTM model has the highest recognition accuracy of depression,reaching 72.24%.It can be seen that in speech depression recognition tasks,deep learning models have more advantages than machine learning models.Bidirectional neural networks can utilize forward and backward feature information,more effectively capture emotional features in speech,and enhance the robustness of the model.(2)Depression recognition based on text.First,the Word2 Vec model is used to vectorize text,and then the Fast Text,Text CNN(Text Convolutional Neural Networks,Text CNN),BERT(Bidirectional Encoder Representations from Transformers,BERT),Bidirectional Long and Short Term Memory Network,and Bidirectional Long and Short Term Memory Network models with Attention mechanism are used to classify text.The comparison revealed that the bidirectional long and short-term memory network model with Attention mechanism performed well in the text depression recognition task,obtaining an accuracy of 78.51%.Therefore,the Attention mechanism can avoid the interference of useless features on the fitting results and improve the accuracy of depression recognition.(3)Depression recognition based on multimodal fusion of text and speech.In this dissertation,the Attention mechanism is used to adjust the weight text and speech features,and then fuse the features.Compared to the results of single mode recognition,the accuracy of depression recognition after bimodal fusion reached 79.57%,which was 7.33% higher than the results of depression recognition for single mode speech,and 1.06% higher than the results of depression recognition for single mode text.(4)Design and implementation of depression recognition system.The function control keys of the depression recognition system are developed by using the Qt Widgets module in the Py Qt framework,and the visualization interface is constructed by using the Qt Gui module.The overall framework includes data reading module,feature visualization module,and depression recognition module.The interface is simple and easy to operate,providing convenient and fast assistance for the diagnosis and treatment of depression.