Research And Application Of Mental Health Detection System Based On Multimodal Physiological And Emotional Characteristics

In recent years,with the rapid development of science and technology and economy,people’s pursuit of material life has gradually changed into the pursuit of a better life,and mental health problems have become a factor restricting the happiness of Chinese people,and gradually become a social problem and a major public health problem that can not be ignored.At present,the mental health scale is often used as the main means of screening detection.However,this method relies heavily on the subjective subjects’ evaluation of self-situation.Due to the influence of social praise effect,subjects often hide their actual situation,resulting in the deviation of the detection results of the psychological scale.The mental health of subjects by doctors is not only highly dependent on doctors and is expensive to diagnose,but also relies too much on the subjective judgment of psychologists and the subjective description of subjects.This subjective diagnosis method is very easy to cause misdiagnosis.In order to address the lack of objective diagnostic methods for mental health status detection,this thesis achieves the detection of mental health status by analyzing objective physiological signals.The main research points of this thesis are as follows.(1)To address the lack of publicly available multimodal mental health data sets,multimodal physiological signals,such as ECG,EEG,EEG,expression signals,and eye movement signals,were collected and recorded from 72 individuals using relevant physiological signal acquisition equipment in the laboratory.The above data were pre-processed and labeled to form the UJNSET-EMO2022 dataset,which provided sufficient data for subsequent experiments and model training.(2)A temporal-spatial dimensional convolutional neural network TS-Attention based on EEG signal features is studied and constructed.The model extracts the spatial dimensional information existing between different channels by using cross-channel convolution in addition to fully retaining the original temporal dimensional information in EEG signals.Compared with the traditional machine learning methods and the current mainstream deep learning frameworks,this network is outstanding in EEG signal processing.(3)A multimodal mental health detection framework,MMHE,has been developed,which makes full use of the rich information carried by multimodal physiological signals,innovatively extracts signal features and emotional features from multimodal physiological signals,and introduces a feature layer evidence fusion mechanism inspired by multi-view evidence fusion theory to further improve the recognition accuracy and interference resistance of MMHE framework.Compared with traditional machine learning algorithms and mainstream neural networks,it has very obvious advantages in accuracy and anti-interference ability in mental health abnormality detection.(4)The system is based on Springboot framework with unimodal physiological signal acquisition module,multimodal physiological signal synchronous and concurrent acquisition module,data processing and labeling module,data visualization module,data analysis module and system management module,which finally realizes the efficient management and analysis of physiological data,realizes the function of multimodal signal real-time acquisition and analysis of mental health status.