Research On Social Information Emotion Recognition Technology Based On Wearable Devices

Mental health issues need to be taken into account as the pressure of life and work is increasing day by day.In traditional mental health diagnosis,the interviewer is required to fill out a psychological questionnaire according to his or her subjective wishes,which is subjective in nature.In order to objectively assess mental health conditions,a new type of assessment method is to use wearable sensors to collect multi-sensory social data from the wearer’s daily life for emotional condition analysis.Most previous studies on emotions are based on laboratory data or emotion-induced data and limited to the classification of emotional delicacy.In thesis,with the goal of detecting abnormal emotions in the daily life of college students,the wearable bracelet was used to collect daily life data and to jointly analyze multimodal sensing signals,focusing on the problem of abnormal emotion recognition in complex signals,and the main research contents are as follows:1.For the problem of abnormal emotion data assessment in natural social states,an experiment of daily wearable multi-sensing information recording was designed.The experiment collected a total of 54 days of daily life data from 27 university student volunteers,which had abnormal emotion information in natural social states.In order to obtain objective abnormal emotion characteristics,volunteers were asked to watch different emotion category videos every day to induce emotional events.2.A Bayesian-based framework for joint analysis of multimodal social information is proposed to address the problem of abnormal emotion misdetection and underdetection.A logical multimodal emotion feature mining method is established through a mapping model architecture between audio,behavioral,and heart rate feature mining and emotion categories guided by mental models,which can effectively distinguish emotion categories.The framework combines features of different modalities through Bayes’ theorem to enhance abnormal emotion recognition.3.To address the problem of complex abnormal emotion recognition in life and social interaction,thesis proposes a logical associative classification feature for abnormal emotion recognition based on the psychology-guided associative model of motor state and environment.The model framework is also integrated into the joint analysis framework to enhance the interpretability of the detection system.The study reached a collaboration with physicians at the Chengdu Mental Health Center,and data from depressed patients were collected for application validation analysis.In thesis,the proposed method is compared and analyzed with other four methods in real social scenarios,including both feature-level fusion methods and decision-level fusion methods.The proposed method outperforms the other four methods in abnormal emotion recognition,and achieves F1＿score of 0.70 and 0.74 for positive and negative abnormal emotions,respectively.