Keypad entry inference with sensor fusion from mobile and smart wearables

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

View graph of relations

Related Research Unit(s)


Original languageEnglish
Article number102837
Journal / PublicationComputers and Security
Online published20 Jul 2022
Publication statusPublished - Oct 2022


This paper investigates the threat of password leakage from sensors on mobile phones and smart wearables. We investigate attack methods for recovering a short random number entered on PEDs using a combination of data collected by from microphone, accelerometer and gyroscope in devices on the person entering the number. We take into consideration features based on keypress sounds, from either phone and/or smart watch, in addition to hand movement acceleration and angular velocity captured by the smart watch. We used the fusion features from these three sensor sources to train a Neural Network to recover key entries on a keypad. Our method based on linear acceleration and angular velocity, with acoustic data used to segment individual keystrokes, showed an improved 74% probability to predict a single keystroke in one guess and 52% for a 6-digit PIN in three guesses.

Research Area(s)

  • Payment, Pin entry devices, PIN recovery, Sensors, Side-channel attack