Video Techniques for Remote Photoplethysmography


Student thesis: Doctoral Thesis

View graph of relations


  • Litong FENG

Related Research Unit(s)


Awarding Institution
Award date16 Feb 2016


Photoplethysmography (PPG) is a low-cost and non-invasive technique to measure the cardiovascular blood volume pulse (BVP). Vital signs, including heart rate (HR), respiration rate, and blood pressure, can be estimated from PPG signals. The traditional contact PPG (cPPG) needs to be attached to a subject’s skin with a probe containing a dedicated light source and a photodetector. Remote PPG (rPPG) is an innovative remote imaging technique which can contactlessly measure BVP information from skin only using ambient light and a digital camera. With the rPPG technique, the measurement of human vital signs becomes very convenient in daily life, because BVP information can be remotely sensed from human face or other skin regions using smartphone cameras or webcams. rPPG is also suitable for long-term monitoring in an unobtrusive and comfortable manner without cabling and clutter. In some special situations, such as monitoring neonates, monitoring burn patients, and sleep studies, rPPG is especially useful to avoid the long-term use of spring-loaded clips of cPPG which can cause pain and skin irritation.
The noncontact operation mode of rPPG also brings a challenging problem. Signal quality of rPPG is much lower than that of cPPG, especially when a low-end digital camera is utilized. Without the aid of a dedicated light source and a custom photodetector, rPPG is susceptible to motion artifacts, region of interest (ROI) fluctuation, and environmental noises. In order to provide reliable rPPG signals for clinical applications, specialized signal/video processing techniques need to be developed to enhance raw rPPG signals. Since rPPG is a video-based BVP measurement technique, video techniques for improving signal quality of rPPG will be investigated in this thesis.
Firstly, the motion-resistance of rPPG is investigated based on the optical properties of skin. A rPPG signal originates from the radiant intensity variation of human skin with pulses of blood, and motions can modulate the radiant intensity of skin. An optical rPPG signal model is built, in which the origins of the rPPG signal and motion artifacts can be clearly described. The ROI of skin is regarded as a Lambertian radiator, and the effect of ROI tracking is analyzed from the perspective of radiometry. By considering a digital color camera as a simple spectrometer, an adaptive color difference operation between the green and red channels is proposed to reduce motion artifacts. Based on the spectral characteristics of PPG signals, an adaptive bandpass filter is proposed to remove residual motion artifacts of rPPG. In addition, ROI selection on the subject’s cheeks is combined with Speeded-Up Robust Features (SURF) points tracking to improve the rPPG signal quality. This motion-resistant rPPG approach can access reliable rPPG signals from moving subjects, thus HR can be estimated accurately.
In addition to the motion artifacts reduction, ROI selection for rPPG is also investigated to further improve rPPG signal quality. rPPG utilizes a camera to capture a video of a skin area especially the facial area, then focuses on a particular sub-area as the ROI. A novel adaptive ROI (AROI) approach is proposed, in which block-based spatial-temporal division is performed on a captured face video. Based on these segmented video pipelines, the spatial-temporal quality distribution of rPPG signals on a face is estimated using a signal-to-noise ratio (SNR) feature. Afterwards, AROIs are calculated through mean-shift clustering and adaptive thresholding in SNR maps. As the AROI can be dynamically adjusted according to the spatial- temporal quality distribution of rPPG signals on the face, the quality of the final recovered rPPG signal is improved. The performance of the proposed AROI approach is evaluated with both still and moving subjects. Compared to the state-of-the-art ROI methods for rPPG, the proposed AROI could obtain a higher accuracy in HR measurement. And the state-of-the-art rPPG enhancement approaches can be effectively improved through being integrated with the AROI.
As a noncontact video-based BVP measurement technique, rPPG is promising for face liveness detection, which is critical for security of a face authentication system. It is assumed that only genuine human faces contain clear rPPG signals with BVP information, while fake human faces cannot provide effective rPPG signals. Based on this assumption, a rPPG-based face anti-spoofing approach is proposed. Spectral entropy (SE) is utilized to define rPPG-based liveness features. The spatial distribution of rPPG-based liveness features is fed into a support vector machine classifier to perform face liveness detection. On a private face anti-spoofing database consisting of 40 subjects, an equal error rate (EER) of 7.50% is achieved by the proposed rPPG-based face anti-spoofing approach, which outperforms the famous multi-scale local binary pattern (LBP) method. When the rPPG-based approach is combined with the LBP approach through feature fusion, an EER of 3.33% can be achieved.