GestOnHMD : Enabling Gesture-based Interaction on Low-cost VR Head-Mounted Display
Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Pages (from-to) | 2597-2607 |
Journal / Publication | IEEE Transactions on Visualization and Computer Graphics |
Volume | 27 |
Issue number | 5 |
Online published | 22 Mar 2021 |
Publication status | Published - May 2021 |
Link(s)
Abstract
Low-cost virtual-reality (VR) head-mounted displays (HMDs) with the integration of smartphones have brought the immersive
VR to the masses, and increased the ubiquity of VR. However, these systems are often limited by their poor interactivity. In this paper, we
present GestOnHMD, a gesture-based interaction technique and a gesture-classification pipeline that leverages the stereo microphones
in a commodity smartphone to detect the tapping and the scratching gestures on the front, the left, and the right surfaces on a mobile
VR headset. Taking the Google Cardboard as our focused headset, we first conducted a gesture-elicitation study to generate 150
user-defined gestures with 50 on each surface. We then selected 15, 9, and 9 gestures for the front, the left, and the right surfaces
respectively based on user preferences and signal detectability. We constructed a data set containing the acoustic signals of 18 users
performing these on-surface gestures, and trained the deep-learning classification pipeline for gesture detection and recognition. Lastly,
with the real-time demonstration of GestOnHMD, we conducted a series of online participatory-design sessions to collect a set of
user-defined gesture-referent mappings that could potentially benefit from GestOnHMD.
Research Area(s)
- Acoustics, Gesture, Google Cardboard, Headphones, Internet, Microphones, Mobile VR, Pipelines, Sensors, Smart phones, Smartphone, Virtual Reality
Citation Format(s)
GestOnHMD : Enabling Gesture-based Interaction on Low-cost VR Head-Mounted Display. / Chen, Taizhou; Xu, Lantian; Xu, Xianshan et al.
In: IEEE Transactions on Visualization and Computer Graphics, Vol. 27, No. 5, 05.2021, p. 2597-2607.Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review