Fully Automatic Non-contact Vital Sign Monitoring Based On Deep Learning

Remote-PhotoPlethysmoGraphy(rPPG)has attracted a wide range of research interests over the past decade,as it is easy to achieve uninterrupted and continuous vital sign monitoring.The principle of rPPG is to measure the faint color change on the skin surface caused by changes in blood volume by a camera.As a source of psycho-physiological information,heart rate is one of the most basic four vital signs of human beings.Remote heart rate monitoring is usually a priority for some practical places(such as hospitals,retail stores,gyms,etc.).Although rPPG can achieve non-contact heart rate monitoring by using different methods such as Principal Component Analysis(PCA)and Independent Component Analysis(ICA),in the step of skin detection,these methods widely use conventional face detection,human Face tracking,skin selection.Continuous multi-step skin selection affects the real-time and accuracy of monitoring,while limiting the detection site.In addition,the cameras currently required to obtain the original rPPG signal are generally expensive,which limits the large-scale application of non-contact heart rate monitoring.In view of the limitations of existing skin detection methods in rPPG heart rate monitoring,in this paper,a universal solution is proposed: using convolutional neural network to detect skin area,combined with a single channel rPPG algorithm for low cost cameras to achieve heart rate signal monitoring in an unconstrained environment.The paper created a benchmark database(including 120 face videos of 43 subjects)to verify the effectiveness of the proposed solution.In addition,the comparative experiments from two aspects of camera hardware conditions and monitoring environment were also set up to evaluate the rPPG signal qualit y of different cost cameras(price less than 600 yuan)with signal-to-noise ratio as an indicator,thus providing a systematic benchmark evaluation method for testing and selecting low-cost cameras.Compared with the widely used three-step skin detection method,the convolutional neural network method(CNN/FCN)used in this paper can complete skin selection in one step to enhance the robustness of heart rate monitoring.Under the mode of manually selecting skin regions,the video quality adopted by the low-cost camera can meet the requirements of a single-channel rPPG heart rate monitoring algorithm.Combining the two,the final experimental results show that using CNN/FCN-rPPG heart rate detection method,a specific low-cost camera has a high signal-to-noise ratio and can be effectively used for large-scale deployment of non-contact heart rate monitoring,which is conducive to the large-scale deployment of non-contact heart rate monitoring.