Research On Measuring Depression Symptoms From Acoustic Features And Linguistic Features In Clinical Interviews

With the intensification of competition in modern society,the number of people who suffer from mental illness,especially depression,continues to rise.However,the current methods of diagnosing depression are time-consuming and labor-intensive,leading to difficulties in medical treatment and a high rate of misdiagnosis and missed diagnosis.Depression diagnosis based on speech signals has the advantages of non-invasiveness,low cost and easy access,and few restrictions on portability.Our research goal is to evaluate depression symptoms from acoustic information and semantic information in speech signals.The main research contents are as follows.Firstly,a hierarchical attention temporal convolutional network model is proposed for depression recognition from acoustic features.For acoustic feature learning on sentence level,area attention is introduced to extract multi-scale sentence features;for segment acoustic feature extraction,simple attention mechanism is applied,which is in line with human cognitive mechanism.For the imbalance problems of positive and negative samples in depression diagnosis,a periodic focal loss function is introduced.Experiments on DAIC-WOZ show that acoustic depression recognition model proposed in this paper show better performance compared with other methods.At the same time,the influence of noise on acoustic depression recognitioin in the real environment is discussed,and our method of adding noise for data augument proves to be effective.Secondly,a hierarchical semantic depression recognition model combining LIWC features and BERT features is proposed for recognizing depression from linguistic features.For sentence-level semantic feature extraction,BERT is introduced for fine-tuning to obtain deep sentence semantic features;for segment-level semantic learning,the learned sentence semantic vector sequences are fed into attention temporal convolutional network and at the same time,LIWC statistical features are extracted from the segments.Experiments show that BERT features perform better than other methods,and the combination of BERT features and LIWC features can improve performance of depression recognition.Then,for the research on acoustic and semantic fusion,a multi-modal fusion depression recognition model is proposed,and feature-level fusion and decision-level fusion are both used to improve accuracy and robustness at the same time.For feature-level fusion,tensor fusion is introduced to capture temporal interactions within modality and between modalities at the same time and a new low-rank decomposition method is adopted to reduce the computational complexity.For decision-level fusion,in view of the absence of a certain modality or excessive noise in real environments,an adaptive weighting method for decision-level fusion is proposed to combine the results of the single-modal models and the tensor fusion model.Experiments on DAIC-WOZ show that multimodal fusion method proposed in this paper shows better accuracy and robustness.Finally,based on the models to evaluate depression symptoms proposed in this paper,an automatic depression evaluation platform based on speech is developed.It can not only be used in hospital or psychological clinics to assist non-professional psychiatrists in diagnosing depression,but also can be used for online artificial intelligence consultations to assess patients’ mental status,which is of great importance and practical.