Tracking 3D Hand Pose Using Wearable Armband

In this project, we investigate a wearable-based technique to track the 3D hand pose of 14 hand skeleton points using the Electromyography (EMG) and motion sensor data from a commercial wearable armband only. This tracking system is not intended to be designed by employing machine learning algorithms to train it on a specific set of predefined hand gestures. We instead strive to track the movement of 3D hand skeleton continuously over time, which can thus serve as a generic and portable platform for the hand-pose oriented applications with new potentials.To track hand poses, existing methods mainly deploy external devices, e.g., cameras and depth sensors, or paste many sensors on forearm by assuming no arm movements. Our technique adopts all the sensory data from only one armband (for one hand), and it is intended to be portable and lightweight to execute on user’s smartphone directly to support upper-layer applications. Recently, researchers find that EMG sensors can measure muscles’ electrical activities and they can be integrated into small devices. Hence, many lightweight EMG-based armbands and methods appear. However, these designs are still limited to recognize a set of pre-defined hand gestures, which cannot track continuous hand-pose motions for applications yet.This project plans to take one step further to track 3D hand poses using the same set of EMG data from commercial armband, while encounters two main challenges. First, armband’s EMG sensors inevitably collect mixed EMG signals from multiple forearm muscles because of the fixed sensor positions on the device, but the prior bio-medical hand-pose models are built on the isolated EMG signal inputs collected from different forearm spots for different muscles. In this project, we will propose a novel wearable sensing based solution to fulfill the hand-pose tracking still using armband's EMG data as input. Second, even the hand pose could be tracked well finally, the system is still not immediately usable if two practical issues are not addressed, including the system setup overhead and the position difference in how the user wears armband each time. In this project, we will propose effective countermeasures to enhance the system’s applicability significantly.In summary, the idea to investigate a wearable-based hand-pose tracking design is of essential novelty, which could offer a fully portable and ubiquitous solution and allow users to bring along the hand-pose tracking ability, without deploying external cameras or depth-sensors. Moreover, technical contributions of our wearable sensing designs are also innovative and useful.