WR-Hand : Wearable Armband Can Track User's Hand

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

13 Scopus Citations
View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number118
Journal / PublicationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume5
Issue number3
Online published14 Sept 2021
Publication statusPublished - Sept 2021

Abstract

This paper presents WR-Hand, a wearable-based system tracking 3D hand pose of 14 hand skeleton points over time using Electromyography (EMG) and gyroscope sensor data from commercial armband. This system provides a significant leap in wearable sensing and enables new application potentials in medical care, human-computer interaction, etc. A challenge is the armband EMG sensors inevitably collect mixed EMG signals from multiple forearm muscles because of the fixed sensor positions on the device, while prior bio-medical models for hand pose tracking are built on isolated EMG signal inputs from isolated forearm spots for different muscles. In this paper, we leverage the recent success of neural networks to enhance the existing bio-medical model using the armband's EMG data and visualize our design to understand why our solution is effective. Moreover, we propose solutions to place the constructed hand pose reliably in a global coordinate system, and address two practical issues by providing a general plug-and-play version for new users without training and compensating for the position difference in how users wear their armbands. We implement a prototype using different commercial armbands, which is lightweight to execute on user's phone in real-time. Extensive evaluation shows the efficacy of the WR-Hand design.

Research Area(s)

  • Deep Learning, Human Hand Pose Construction, Mobile Sensing