| As an important part of human-computer interaction,emotion recognition has broad application prospects in many fields such as distance education,health care,and humancomputer interaction.The vital sign signals have the advantages of being objective and can hardly be masked,which have gradually become a hot spot in the research of emotion recognition.Existing emotion recognition systems based on vital sign signals are mostly based on contact sensors,which cause discomfort to the subjects.Moreover,the existing non-contact emotion recognition systems using single sensor based on vital sign signals have the problem of low recognition accuracy Aiming at the above problems,this paper proposed a non-contact bimodal emotion recognition system based on radar and video sensor.The system can optimize the signal according to the advantages of each sensor to achieve high accuracy recognition of emotion.1.The composition and working principle of the non-contact bimodal emotion recognition system was introduced.We analyzed and verified the human respiratory signals and heartbeat signals acquired by the continuous wave radar sensor and the human heartbeat signals acquired from the video sensor.2.Aiming at the problem of poor performance of video heartbeat signal detection in weak light environment,a heartbeat signal optimization algorithm based on light intensity was proposed.This algorithm can select the source of higher accuracy heartbeat signal under different light intensities,which can effectively improve accuracy of heartbeat signal detection in the weak light environment.To solve the problem that the vital signs obtained by radar were susceptible to body motion,a motion detection algorithm based on the video optical flow signals was proposed.The algorithm can effectively detect the body motion segment in the vital signals and eliminate the influence of body motion on the acquisition of the radar signal.3.The time domain,frequency domain,nonlinear and geometrical features were extracted form optimized signals.We designed the data fusion models of bimodal feature level and decision level.the related machine learning algorithm was used for emotion recognition.4.The emotion recognition experiment was designed to obtain the physiological signals of the subjects under four emotional states of happiness,sadness,calmness and fear.The experiments on single sensor,optimized algorithm and fusion algorithm were carried out.The experimental results showed that the accuracy of emotion recognition for single radar sensor was 73.7%,the accuracy of emotion recognition for single video sensor was 54.8%,the accuracy of radar module recognition after optimized algorithm was 78.6%,and the recognition accuracy of video module was 66%.After the fusion algorithm,the recognition accuracy of decision-level was 81.4%,and recognition accuracy of feature-level recognition accuracy was 82.8%. |
