Research On Non-Invasive Stress Recognition Based On Multimodal Collaborative Expression With Attention Mechanism

Psychological stress is increasing due to the development of society,fierce competition,and increasing life demands.It is essential to study how to assess individual stress as prolonged periods of high psychological stress can lead to worsening health problems.Research on emotional stress recognition has progressed from single modality to multimodal and from contact to non-perturbative.However,most non-perturbative stress recognition studies focus on a single modality.This paper aims to combine camera technology with pulse wave signals and facial expression features to achieve emotional stress recognition.The main work of this paper is as follows:(1)To address the problem of large heart rate estimation errors caused by head movements during video acquisition,this paper proposes a two-way recurrent neural network-based optical flow facial alignment module.This module analyzes facial motion information in consecutive frames and corrects face angle offset frames to the target frame,which is directly opposite to the camera.This ensures the effectiveness of feature extraction.(2)This paper proposes a multi-region multi-head attention mechanism for pulse wave denoising to address the low signal-to-noise ratio in face-based pulse wave extraction.The proposed mechanism effectively removes interference information,resulting in a purer pulse wave signal.To address poor model generalization,this study uses synthetic original signals for training data enhancement.The results show that the proposed method performs well in the extraction of long-time dependencies and outperforms traditional r PPG techniques in denoising periodic noise in the source signal,improving the Pearson correlation coefficient by 0.12.(3)This study proposes a stress recognition network based on a multi-scale attention mechanism and Transformer high-order interaction fusion to address the issue of different information contained in features of different scales.Facial expression and pulse wave features are extracted using CNN and one-dimensional convolutional net-works and sent to a multi-scale attention layer to learn the weight of each scale.An Interaction-Attention(ITA)mechanism is introduced for modality fusion,which introduces a fusion vector as a medium for information exchange between modalities.Experimental results show that the proposed model improves information processing efficiency while reducing computational complexity.(4)This paper validates the proposed method’s accuracy and robustness by constructing a local dataset and conducting an emotion-induced stress experiment.Heart rate signals were recorded during facial video collection,and the proposed method was evaluated in experiments on facial alignment,pulse wave denoising,and multi-modal fusion classification using this dataset.The results demonstrate the effectiveness of the proposed method.In summary,This paper proposes a method for extracting pulse wave signals from facial features and evaluating psychological stress through multi-modal fusion.The study includes preprocessing steps such as facial video alignment and pulse wave signal denoising,as well as exploring the significance of multi-scale self-attention learning.Through experiments,the proposed method achieves a stress recognition accuracy of 89.47%,which is 4.05% higher than the TSDNet stress recognition network.This study contributes to the advancement of non-invasive emotion-induced stress recogni-tion technology.