Research on Sensing Technologies of Smart Health Applications based on Wearable Devices


Student thesis: Doctoral Thesis

View graph of relations


Related Research Unit(s)


Awarding Institution
Award date25 Apr 2023


Due to the rapid development of the Internet of Things, a large number of wearable devices have emerged, such as smart wristbands, smart watches, smart headphones and so on. These wearable devices bring great convenience to the lives of the deaf and the elderly. In smart health applications based on wearable devices, the sign language translation system builds a strong bridge for communication between deaf-mute people who can sign language and people who don't. However, the current sign language translation system requires users to collect a large amount of data, which greatly increases the burden on users. Another communication bridge is the hearing aids, which made up for the defects caused by hearing loss of the hearing impaired people. However, these hearing aids on the market do not have mature security solutions, which makes users have security risks of privacy leakage when using them. Therefore, there are still a series of applicability issues before these auxiliary applications for hearing impaired people are actually put into production. In this thesis, we strive to first identify and solve these challenging issues, so that the usability of these applications can move a step further.

First, we find that there are words and sentences in the sign language translation systems, previous works usually collects word and sentence data separately, which greatly increases the burden on users. Therefore, we analyze the IMU sensor data of the smart watch, and we observe that the word data can be concatenated into sentence data. The key challenge is that there are only IMU sensors embedded in smartwatch, and the accelerometer data can not be used to complete the design due to the severe data drift. Thus, we investigate the feasibility of the orientation data and find that the orientation data can support this design. Leveraging this observation, we design a set of algorithms to craft sign language sentence data from collected word data and develop a sign language translation system based on smartwatch. The experiment results demonstrate the effectiveness of the proposed algorithms, which can greatly release the overhead of those sign language users.

Second, according to our survey, more and more earable devices have been integrated heart rate sensors, which are actually PPG (Photoplethysmography) sensors. The signals collected from PPG sensors can be used not only to measure key physiological indicators, but also to act as a way of biometric authentication. Therefore, we propose a PPG sensor-based authentication system as a security solution for today's hearing aids. In particular, we design an earable prototype based on PPG sensor and leverage the uniqueness of human cardiac system to conduct user authentication. The key challenge stems from the motion artifacts due to daily activities. Thus, we propose a set of algorithms to handle the motion artifacts to advance the usability and guarantee the security of current hearing aids and earphones.

Third, we discuss and identify four possible research opportunities. Firstly, we plan to leverage more advanced deep learning techniques to enable few shot sign language translation. Secondly, we consider combining with advanced hardware design to enable more powerful earable applications. Thirdly, we consider leveraging the PPG sensor to develop more intelligent applications. Finally, we propose two more designs to further enhance the applicability of the sign language translation systems.

In summary, this dissertation studies the applicability issues which exists in current smart health applications, and propose a series of techniques to address these issues. Our research outcome could also enrich the methodologies in the wearable sensing field and is beneficial to the design space and the development prospects of future smart health applications.