Research On Depression Recognition Method Based On Facial And Eye Landmark Motion Analysis

Depression is a common mental disorder,with over 264 million people living with depression worldwide,according to a 2018 Lancet study.However,misdiagnosis and lack of treatment resources are widespread problems.With the advancement and refinement of deep learning techniques,more and more researchers are proposing to use deep learning algorithms for the purpose of objectively and effectively identifying depression.One of the hot research directions is depression recognition based on facial video images from clinical interviews.In this project,we have conducted a study on how to achieve convenient and efficient recognition of depression using clinical interview videos,and the main innovations and contributions of this paper are as follows.First,considering the problems of mask occlusion and video images being greatly affected by lighting and environmental changes,this paper uses landmark detection techniques to pre-process the interview videos and realizes a facial and human eye landmark detector that can overcome the mask occlusion problem.Then,according to clinical knowledge,prolonged depressed mood is the main outward manifestation of depression.In this paper,with emotional states being classified into three categories of emotional labels: Positive,Neutral and Negative,the external representation patterns of different emotional states are investigated,and sentence-level embeddings are proposed to be learnt from temporal landmark sequences to achieve the emotional state recognition task.For the problem that uncontrolled factors such as occlusion and head movement cause the intra-class distance of emotional states to increase and the inter-class distance to decrease,this paper uses the A-Softmax loss function to define a large-angle interval learning task with adjustable difficulty.Theoretically and experimentally,it is demonstrated that this loss can effectively reduce the intra-class distance and increase the inter-class distance of emotional states compared with the Softmax loss.Finally,in view of the shortcomings of the emotion recognition model’s insufficient utilization of the spatial dimension information of the landmark sequence,this paper proposes a depression state recognition algorithm using spatial temporal graph convolution neural network.In this work,we utilize the spatial temporal graph,which is constructed by the facial and eye landmark sequence,to form hierarchical representation of the landmark sequences.Considering the close correlation between the emotional state and the depression a dual-task learning strategy of emotional state recognition and depression recognition is proposed.Experiments show that:(1)the spatial temporal graph of landmark sequence can effectively learn both the spatial and temporal patterns from data;(2)the dual-task learning strategy of emotion and depression recognition can effectively improve the performance of the depression recognition model.Last but not least,the depression recognition model proposed in this paper achieved F2 scores of 84.8% and 83.3% on the self-built database 301 database and the public DAIC-WOZ depression database,respectively.