Research On Face Video Heart Rate Detection Technology Based On Deep Learning

As the most important vital sign of human body,heart rate has been paid more attention with the improvement of life quality.Now,the widely used heart rate detection equipment has limitations in use,which can not achieve continuous monitoring.In contrast,the non-contact heart rate detection based on face video has more advantages and wider application prospects.Most recent work focus on the heart rate monitoring in simple scenes,such as static state with slight head movement and light changes.In the complex scenarios including face rotation and facial expression,the extracted r PPG signal is interfered by motion noise.It is desired for further investigation.At the same time,the deep learning framework is adopted in the non-contact heart rate measurement for better accuracy.This paper focuses on the face video heart rate detection technology based on deep learning.The work mainly includes the following issues:(1)Dlib is used to detect the key points of human face.With face detection,a prejudgment module is added to optimize the ROI region selection.It effectively reduces the influence of background noise introduced by the face detector in rotation cases.Besides,the results show that the signal-to-noise ratio of the r PPG signal extracted from the cheek region is the highest.(2)The raw r PPG signal extracted from ROI is preprocessed.Firstly,the influence of the uneven time interval between the frames captured by the camera on the r PPG signal is analyzed,and the uniformly sampled signal is reconstructed by linear interpolation and then extracted by equal interval.Secondly,L1 trend filtering is used to remove the interference signal other than the trend component and the corresponding frequency of heart rate in the reconstructed r PPG signal.Finally,the phase tracking module is added to optimize the detrended filtered signal into sinusoidal like signal,which makes the periodicity of the signal stronger in the time domain and the frequency corresponding to the heartbeat more prominent in the frequency domain.(3)For the preprocessed r PPG signal,feature images are generated in both time domain and time-frequency domain respectively,and convolution neural network is constructed for training and prediction.In the time domain,the r PPG signals extracted by three different algorithms are regarded as separate channels of the image and superimposed.In the time-frequency domain,the time-frequency map corresponding to the signal is regarded as the feature map of the r PPG signal.The experimental results show that the best prediction effect is achieved when the attention mechanism is added to Res Net18 network using time domain feature map,and the performance is MAE=4.81,RMSE=6.44 and Pearson correlation coefficient ?=0.86.It is verified that the uniform resampling in preprocessing improve the prediction performance by6.59 in terms of MAE and 7.67 in terms of RMSE.Also,it is shown that the length of signal segments has slight influence on the performance.