Contactless Physiological Health Sensing: Challenges, Solutions & Opportunities

Contactless health monitoring techniques, such as Remote Photoplethysmography (rPPG) [1] and video-based Respiratory Rate (RR) [2] estimation, have emerged as promising methods utilizing regular camera sensors for capturing vital signs like Heart Rate (HR) and Respiratory Rate (RR). These approaches offer a cost-effective, widely applicable, and safe solution for regular and long-term health monitoring. However, the primary challenge lies in extracting accurate vital signals from the captured videos due to their low signal-to-noise ratios. Traditional signal processing analyzes temporal properties of spatial regions of interest across video frames to extract vital signs, while computer vision-based Deep Learning (DL) approaches leverage large-scale datasets and computational power for data driven decision-making without heuristic reliance [3]. In this tutorial, we aim to provide a comprehensive background on rPPG and video-based heart rate estimation, covering signal acquisition and physics-inspired estimation principles. We will discuss signal-processing approaches and their limitations. We will delve into DL-based estimation systems, addressing the challenge of inherent aleatoric uncertainty (irreducible uncertainty from various stochastic factors like sensor variations and inter-subject differences) in the ground truth data annotation that hinders the development of generalized deep learning-based rPPG estimation system. To reinforce a robust DL model addressing these inherent uncertainties in rPPG data streams, we will discuss three novel deep-learning approaches. First, a multi-task learning method for rPPG estimation that learns the core rPPG features using shared embedding across noisy ground truth by separating individual targets. Second, a self-supervised learning method that leverages unlabeled rPPG data to learn the intrinsic rPPG properties and improve rPPG feature representation and signal reconstruction by incorporating the prior domain knowledge (HR frequency, phase, and temporal coherence). Third, a generative adversarial method that enables tine-grained rPPG learning without direct supervision using large-scale rPPG data. Finally, we will present the validation of the proposed methods across in-house MPSC-rPPG datasets, and multiple public datasets with open-source code for further exploration. Furthermore, we will demonstrate a real-time heart rate estimation system, RhythmEdge, using a low-cost camera sensor and describe the rPPG-specific pruning techniques to reduce rPPG model size for efficient edge implementation. We will conclude the tutorial with existing research gaps and potential directions in contactless rPPG and respiratory rate estimation research.