Dynamic Liveness Detection Based on Fusion of mmWave Radar and Vision

Liveness detection is critical to various scenarios such as autonomous vehicles. A remaining challenge of this topic is accurately identifying liveness from other visual disruptions, such as distinguishing persons from printed characters in billboard advertisements or LED screens in real-world scenarios. To address this problem, we leverage multi-model information, i.e., millimeter wave (mmWave) and vision, to improve the accuracy and robustness of liveness detection. We propose a feature fusion network grounded on the attention mechanism, which amalgamates mmWave radar features with visual features to augment live object detection. To validate our approach, we collect a multi-model liveness detection dataset using commercial-off-the-shelf mmWave radar and camera. We then evaluate the effectiveness and robustness via this real-world dataset. Results show that our approach could achieve 13.5%–45.6% improvement on the mAP metric compared with the state-of-the-art vision-based techniques. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.