Research On Emotion Recognition Based On Millimeter Wave Radar

Emotions play a very important role in people’s daily lives.With the continuous development of Internet of Things technology and artificial intelligence technology,automatic recognition of emotions as an important part of artificial intelligence has a wide range of application scenarios in many fields,including human-computer interaction,healthcare,education,multimedia entertainment,and traffic safety.Currently,emotion recognition based on speech,image,and physiological signals has achieved numerous achievements.Especially,compared with image and speech signals,physiological signals have the characteristic of not being easily affected by subjective human factors and can be used as a more objective identifier for emotion recognition.However,traditional emotion recognition based on physiological signals collects the signals through contact sensors attached to the human body.In the process of signal collection,the close contact between the sensors and the human body causes discomfort and negative effects on their emotions,thereby introducing additional errors into emotion recognition.Moreover,this close contact between humans and sensors makes this method of physiological signal collection unsuitable for deployment in daily life.Therefore,the paper proposes a method of using industrial millimeter-wave radar to obtain human reflection signals and extract corresponding heartbeat signals in a non-contact manner,and then conduct emotion recognition,providing a new solution for non-contact physiological emotion recognition.The specific research content and experimental results of the paper include the following two points:(1)Create a millimeter-wave radar-based physiological database for emotions.Affective induction experiments were designed using videos as emotional stimuli.Existing industrial millimeter-wave radar was used to obtain radar reflection signals from the human body under different emotional states,and through a series of data processing algorithms,corresponding radar heartbeat signals were obtained.First,a human positioning algorithm based on 4D beam scanning was proposed to determine the three-dimensional coordinates of the human body from the radar reflection signals and extract the human reflection signals.Then,a heartbeat detection algorithm based on second-order differentiation and continuous heartbeat self-similarity was proposed to extract the time-domain signals that can characterize the mechanical activity of the heart from the human reflection signals,and segments were obtained containing single heartbeat beats to measure the heart rate variability.The accuracy was comparable to the RR interval obtained by contact sensors collecting electrocardiogram signals,and the average error of heart rate intervals was 4.3ms.The experiment collected radar reflection signals under three discrete emotional states:calm,happy,and fearful,including more than 40,000 heartbeat intervals from 72 college students.Synchronized electrocardiogram signals were collected using Shimmer3 sensor as reference signal.(2)Construct an emotion recognition model based on millimeter-wave radar heartbeat signals.Emotional-related features were extracted from the radar heartbeat signals,including heart rate variability features and heartbeat signal morphology features.Three binary classification tasks and one three-classification task were performed for the calm,happy,and fearful three discrete emotions using K-nearest neighbor algorithm,support vector machine and random forest machine learning classifiers,and compared and analyzed with the classification results based on traditional ECG signals.The experimental results of ten-fold cross-validation showed that emotion recognition based on radar heartbeat signals achieved a classification effect that can be compared with or even better than emotion recognition based on ECG signals.The K-nearest neighbor classifier achieved the best classification effect,with binary classification accuracies of 92.40%,89.87%,and 88.72% for calm,happy,and fearful emotions,and a three-classification accuracy of 83.60%.These four classification results were higher than the classification results of the ECG signal,verifying the feasibility of emotion recognition based on millimeter-wave radar.Compared with heart rate variability features,the heartbeat signal morphology features exhibited better classification effects.The average binary classification accuracy of K-nearest neighbor,support vector machine,and random forest in the heartbeat signal morphology feature set were 90.33%,82.83%,and 84.98%,respectively,while in the heart rate variability feature set,the average recognition rates were 60.62%,67.40%,and 65.55%,respectively.The three classifiers’ average classification accuracy in the heartbeat signal morphology feature set was higher than their average recognition rate in the heart rate variability feature set by 29.71%,15.43%,and 19.43%,respectively.This indicates that compared with heart rate variability features,heartbeat signal morphology features are more suitable for emotion recognition based on radar heartbeat signals.The research results of the paper show that emotion recognition based on millimeter-wave radar is feasible.The three classifiers used achieved a classification effect that can be compared or even better than emotion recognition based on traditional ECG signals.This research provides a new solution for non-contact physiological emotion recognition.