Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

Chunki Yiu; Yiming Liu; Wooyoung Park; Jian Li; Xingcan Huang; Kuanming Yao; Yuyu Gao; Guangyao Zhao; Hongwei Chu; Jingkun Zhou; Dengfeng Li; Hu Li; Binbin Zhang; Lung Chow; Ya Huang; Qingsong Xu; Xinge Yu

doi:10.1126/sciadv.adt6041

Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

Chunki Yiu (Co-first Author), Yiming Liu (Co-first Author), Wooyoung Park (Co-first Author), Jian Li, Xingcan Huang, Kuanming Yao, Yuyu Gao, Guangyao Zhao, Hongwei Chu, Jingkun Zhou, Dengfeng Li, Hu Li, Binbin Zhang, Lung Chow, Ya Huang, Qingsong Xu^*, Xinge Yu^*

^*Corresponding author for this work

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

18 Citations (Scopus)

23 Downloads (CityUHK Scholars)

Abstract

Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone’s posture into two-dimensional tactile information, enhancing the operator’s perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation–based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone’s posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design. Copyright © 2025 The Authors, some rights reserved.

Original language	English
Article number	eadt6041
Journal	Science Advances
Volume	11
Issue number	13
Online published	26 Mar 2025
DOIs	https://doi.org/10.1126/sciadv.adt6041
Publication status	Published - Mar 2025

Funding

This work was supported by the National Key R&D Program of China (grant no. 2024YFB4707503 to X.Y.), the Innovation and Technology Fund of Innovation and Technology Commission (grant no. ITS/119/22 to X.Y.), the Shenzhen Science and Technology Innovation Commission (grant no. SGDX20220530111401011 to X.Y.), the Research Grants Council of the Hong Kong Special Administrative Region (grant nos. 11211523, 11213721, and 11215722 to X.Y.), the National Natural Science Foundation of China (grant no. 62122002 to X.Y.), the City University of Hong Kong (grant nos. 9667221, 9667246, 9680322, and 9667199 to X.Y.), the InnoHK Project on Project 2.2\u2014AI-based 3D ultrasound imaging algorithm at Hong Kong Centre for Cerebro-Cardiovascular Health Engineering (COCHE) (to X.Y.), and the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (grant no. RS-2024-00411904 to X.Y.).

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

RGC Funding Information

RGC-funded

Access Output

10.1126/sciadv.adt6041

279308860.pdfFinal Published version, 2.26 MBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Yiu, C., Liu, Y., Park, W., Li, J., Huang, X., Yao, K., Gao, Y., Zhao, G., Chu, H., Zhou, J., Li, D., Li, H., Zhang, B., Chow, L., Huang, Y., Xu, Q., & Yu, X. (2025). Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control. Science Advances, 11(13), Article eadt6041. https://doi.org/10.1126/sciadv.adt6041

@article{20ad7beb2cd7408698ff48864da75dc0,

title = "Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control",

abstract = "Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone{\textquoteright}s posture into two-dimensional tactile information, enhancing the operator{\textquoteright}s perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation–based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone{\textquoteright}s posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design. Copyright {\textcopyright} 2025 The Authors, some rights reserved.",

author = "Chunki Yiu and Yiming Liu and Wooyoung Park and Jian Li and Xingcan Huang and Kuanming Yao and Yuyu Gao and Guangyao Zhao and Hongwei Chu and Jingkun Zhou and Dengfeng Li and Hu Li and Binbin Zhang and Lung Chow and Ya Huang and Qingsong Xu and Xinge Yu",

year = "2025",

month = mar,

doi = "10.1126/sciadv.adt6041",

language = "English",

volume = "11",

journal = "Science Advances",

issn = "2375-2548",

publisher = "American Association for the Advancement of Science",

number = "13",

}

Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control. / Yiu, Chunki (Co-first Author); Liu, Yiming (Co-first Author); Park, Wooyoung (Co-first Author) et al.
In: Science Advances, Vol. 11, No. 13, eadt6041, 03.2025.

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

TY - JOUR

T1 - Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

AU - Yiu, Chunki

AU - Liu, Yiming

AU - Park, Wooyoung

AU - Li, Jian

AU - Huang, Xingcan

AU - Yao, Kuanming

AU - Gao, Yuyu

AU - Zhao, Guangyao

AU - Chu, Hongwei

AU - Zhou, Jingkun

AU - Li, Dengfeng

AU - Li, Hu

AU - Zhang, Binbin

AU - Chow, Lung

AU - Huang, Ya

AU - Xu, Qingsong

AU - Yu, Xinge

PY - 2025/3

Y1 - 2025/3

N2 - Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone’s posture into two-dimensional tactile information, enhancing the operator’s perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation–based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone’s posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design. Copyright © 2025 The Authors, some rights reserved.

AB - Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone’s posture into two-dimensional tactile information, enhancing the operator’s perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation–based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone’s posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design. Copyright © 2025 The Authors, some rights reserved.

UR - http://www.scopus.com/inward/record.url?scp=105001336860&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-105001336860&origin=recordpage

U2 - 10.1126/sciadv.adt6041

DO - 10.1126/sciadv.adt6041

M3 - RGC 21 - Publication in refereed journal

C2 - 40138404

SN - 2375-2548

VL - 11

JO - Science Advances

JF - Science Advances

IS - 13

M1 - eadt6041

ER -

Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

Abstract

Funding

UN SDGs

Publisher's Copyright Statement

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Thin, Soft, Skin-integrated Olfactory Interface

GRF: Intelligent Skin-integrated Haptic Interface with Multiple Feedback Modes

ITF: An Intelligent Haptic Glove for Metaverse

Cite this

Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

Abstract

Funding

UN SDGs

Publisher's Copyright Statement

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Thin, Soft, Skin-integrated Olfactory Interface

GRF: Intelligent Skin-integrated Haptic Interface with Multiple Feedback Modes

ITF: An Intelligent Haptic Glove for Metaverse

Cite this