Video-based Blood Volume Pulse Signal Measurement Study With Generative Adversarial Network

Remote photoplethysmography(r PPG)is a non-contact technique for measuring blood volume pulse(BVP)signals from facial videos.It has a wide range of applications in stress detection,emotion classification and health monitoring.High quality BVP signal can be obtained by r PPG to calculate a variety of cardiac-related physiological indicators.However,most of the existing r PPG methods can only be used to get average heart rate values due to the limitation of inaccurate BVP signals.In this thesis,a new framework based on generative adversarial network(GAN),called PulseGAN,is introduced to generate high-quality BVP signals through denoising the rough BVP signals obtained by traditional method.The main contents are as follows:First,the 68 facial landmarks are detected and tracked from the video frame sequence to determine the region of interest(ROI).The rough BVP signal is then extracted from RGB channels within the selected ROI using an existing chrominance model.Next,the proposed PulseGAN network is built,and appropriate loss functions are constructed for training the network.Considering that the BVP signal is quasi-periodic and has apparent time-frequency characteristics,the loss functions in both time and spectrum domains,together with the adversarial loss,are defined to enforce the model accurate pulse waveform generation.Finally,the rough and reference BVP signals are taken to train the network,and the network parameters are adjusted to optimize the model.The optimal model is selected and tested with new data.The results of tests on three public databases,UBFC-RPPG,PURE and MAHNOB-HCI,showed that PulseGAN can effectively improve the BVP waveform quality,thereby enhancing the accuracy of heart rate and heart rate variability.Meanwhile,the loss functions,the ROI and the input signals which affect the performance of PulseGAN are also analyzed to verify the effectiveness of the proposed method.The PulseGAN framework proposed in this thesis can extract high-quality BVP waveform to calculate a variety of heart rate related indicators,thus effectively expanding the application scope of r PPG technology.