Efficientphys: Enabling simple, fast and accurate camera-based vitals measurement

Xin Liu, Brian Hill, Ziheng Jiang, Shwetak Patel, Daniel McDuff

PDF

Abstract

Camera-based physiological measurement is a growing field with neural models providing state-of-the-art performance. Prior research has explored various ``end-to-end'' architectures; however these methods still require several preprocessing steps and are not able to run directly on mobile and edge devices. The operations are often non-trivial to implement, making replication and deployment difficult and can even have a higher computational budget than the ``core'' network itself. In this paper, we propose two novel and efficient neural models for camera-based physiological measurement called EfficientPhys that remove the need for face detection, segmentation, normalization, color space transformation or any other preprocessing steps. Using an input of raw video frames, our models achieve strong accuracy on three public datasets. We show that this is the case whether using a transformer or convolutional backbone. We further evaluate the latency of the proposed networks and show that our most lightweight network also achieves a 33% improvement in efficiency.