Towards Ubiquitous Wearable-based Human-Centered Sensing

面向以人為中心的普適可穿戴感知

Student thesis: Doctoral Thesis

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Awarding Institution
Supervisors/Advisors
Award date3 Jul 2024

Abstract

Wearable devices play significant roles in human-centered sensing and Human-Computer Interaction (HCI) since they enable the acquisition of rich information by sensing human behaviors at close range. Existing works have extensively investigated the wearable devices (e.g., smartwatch, smartphone, ring, EMG armband). However, there are some long-standing challenges in the ubiquitous deployment of wearable sensing, such as the heterogeneity problem of biological signals (e.g., EMG) degrades the performance when deploying such systems across different individuals; the lack of sensing systems that can seamlessly integrate into existing ecosystems, capable of naturally and user-friendly sensing the extensive information around the human head (e.g., EEG, EOG, EMG, facial movements, speech). As a result, human-centered wearable sensing is undergoing a transformative shift toward generalization and fine-grained multimodal sensing. Nonetheless, existing solutions tackle these challenges either through the use of abundant labeled data or specialized hardware. However, the labor-intensive nature and high deployment cost of these solutions significantly limit their widespread adoption.

In this dissertation, we aim to contribute to the ubiquitous deployment of human-centered intelligent sensing using wearable devices. Specifically, we contribute to it from two perspectives: (1) reducing the deployment overhead across individuals caused by biological heterogeneity; (2) proposing a corpus of intelligent applications based on commercial off-the-shelf (COTS) headphones. To achieve such goals, we first formulate the mitigation of performance degradation caused by heterogeneity as an unsupervised multi-source domain adaptation problem to learn the most general features among several existing individuals. Then, by innovatively using self-supervised learning techniques to relearn the user-specific features. Comprehensive evaluations demonstrate that the proposed framework can achieve satisfactory performance and comparable with the model trained in a supervised manner. To bridge the gap in the availability of sensing platforms based on COTS headphones, we carefully design a hardware prototype based on COTS headphones without any circuit modifications. By attaching two auxiliary spacers to one side of the headphones' ear pad, we create a gap allowing ultrasound to "escape" and sweep across the user's face. The escaped ultrasound will be received by a boom microphone. Finally, a stable acoustic sensing field will be established across the user’s face, which enables a corpus of intelligent applications—speech enhancement, voiceprint-based authentication enhancement, and artifact anticounterfeiting.

In summary, this dissertation carefully studies wearable devices supporting human-centered sensing from two perspectives, making significant contributions to addressing the heterogeneity problem of EMG signals and pioneering novel intelligent applications based on earable devices.