基于深度展开与深度平衡模型从视频中恢复脉搏波

Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models

摘要 Abstract

基于相机的生命体征监测(也称为成像光电容积描记法,iPPG)已在驾驶员监控、手术环境中的灌注评估、情感计算等领域得到应用。iPPG 涉及从皮肤视频中感知潜在的心脏脉搏,并估算心率或完整的脉搏波形等生命体征。一些先前的 iPPG 方法对脉冲信号施加基于模型的稀疏先验,并使用迭代优化进行脉搏波恢复,而其他方法则采用端到端的黑盒深度学习方法。相比之下,我们引入了一种结合信号处理和深度学习的方法,将其置于逆问题框架中。我们的方法通过学习基于深度网络的去噪算子,利用深度算法展开和深度平衡模型,从面部视频中估计潜在的脉搏信号和心率。实验表明,我们的方法可以去噪从面部获取的信号并推断正确的潜在脉搏率,在知名基准测试中实现了最先进的心率估算性能,且可训练参数数量不到最接近竞争方法的五分之一。

Camera-based monitoring of vital signs, also known as imaging photoplethysmography (iPPG), has seen applications in driver-monitoring, perfusion assessment in surgical settings, affective computing, and more. iPPG involves sensing the underlying cardiac pulse from video of the skin and estimating vital signs such as the heart rate or a full pulse waveform. Some previous iPPG methods impose model-based sparse priors on the pulse signals and use iterative optimization for pulse wave recovery, while others use end-to-end black-box deep learning methods. In contrast, we introduce methods that combine signal processing and deep learning methods in an inverse problem framework. Our methods estimate the underlying pulse signal and heart rate from facial video by learning deep-network-based denoising operators that leverage deep algorithm unfolding and deep equilibrium models. Experiments show that our methods can denoise an acquired signal from the face and infer the correct underlying pulse rate, achieving state-of-the-art heart rate estimation performance on well-known benchmarks, all with less than one-fifth the number of learnable parameters as the closest competing method.

基于深度展开与深度平衡模型从视频中恢复脉搏波 - arXiv