Technologies to Enhance Smart Health Through Mobile-based Sensing

Abstract

Smart health uses mobile devices and sensing technology to collect and analyze patient health information in real time, allowing doctors to promptly understand and respond to patients' health status. This is crucial for managing chronic diseases and rehabilitation treatments like dyslexia and stroke recovery. In this dissertation, we investigate how pervasive, low-cost mobile devices can augment smart health applications leveraging diverse mobile sensors and advancements in deep learning. First, we design a new gaze tracking system leveraging the common RGB camera from mobile phones, demonstrating how these ubiquitous devices can provide reliable data for eye-related health assessments. In addition to improving the accuracy of existing RGB camera-based gaze tracking methods, a novelty is that it can track gaze points on various surfaces, such as phone screens, computer displays or even non-electronic surfaces like whiteboards or paper - a situation that is challenging for existing methods. To achieve this goal, we propose efficient and practical techniques to address three unique challenges, including errors of iris boundary pixels, ambiguity of the gaze direction, and mapping of gaze points to the tracking surface area. Next, we examine the utilization of wrist-worn electromyography (EMG) sensors for finger tracking. Recent EMG-based finger tracking methods mainly employ specific armbands worn on the user’s forearm. However, this additional cost that users need to purchase an extra armband and wear it on their forearm every time makes their tracking solution still less user-friendly and hinders the widespread adoption of finger tracking technology among the public. To this end, we investigate the feasibility of moving EMG sensors from the forearm to the wrist for finger tracking. As sensor placement varies, we find new challenges in determining good locations to place sensors to gather useful information to capture all finger movements and using low-quality signals to still ensure accurate tracking. To address these challenges, we propose new, efficient solutions, including tracker network and branch adapter based on the valuable insights of feasibility study. Finally, we thoroughly discuss the practical issues that need to be considered when deploying mobile health sensing systems in everyday applications. These issues have motivated us to propose a series of efficient techniques aimed at optimizing our mobile health platform from three key aspects: automatically generating labels for the training dataset, accelerating inference speed on mobile devices, and enhancing system performance. In conclusion, this dissertation presents a series of innovative methodologies for enhancing healthcare applications by harnessing the sensing capabilities of mobile devices.

Date of Award	12 Aug 2024
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Zhenjiang LI (Supervisor)