Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input

Yantao Liu; Dongmin Xiao; Taizhou Chen; Kening Zhu

doi:10.1145/3706599.3720116

Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input

Yantao Liu, Dongmin Xiao, Taizhou Chen^*, Kening Zhu

^*Corresponding author for this work

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Touchscreen typing on tablet has become popular in modern digital routine, calling for investigation of more expressive input method on touchscreen keyboard. In this paper, we propose a novel approach to augment touchscreen typing experience by integrating key-press finger posture recognition to extend the input space of a standard touchscreen keyboard. Our system distinguish between finger tip contact(FP) and finger pad contact(FP) through acoustic sensing, enabling seamless switching between normal and functional input. We evaluate the performance of our system through offline and online experiments, where we show that our system achieves an offline key-wise recognition accuracy of up to 96.3%. The online experiment shows a real-time recognition accuracy of 94% and 88% in quiet and noisy environments, respectively. We further conducted a usability study on text formatting task, which shows that our method significantly outperform the baseline method in terms of input speed and functionality. © 2025 Copyright held by the owner/author(s).

Original language	English
Title of host publication	CHI EA '25
Subtitle of host publication	Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
Editors	Naomi Yamashita, Vanessa Evers, Koji Yatani, Xianghua (Sharon) Ding
Publisher	Association for Computing Machinery
ISBN (Print)	979-8-4007-1395-8
DOIs	https://doi.org/10.1145/3706599.3720116
Publication status	Published - Apr 2025
Event	2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025): IkiCHI - PACIFICO Yokohama, Yokohama, Japan Duration: 26 Apr 2025 → 1 May 2025 https://chi2025.acm.org/

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025)
Abbreviated title	CHI'25
Country/Territory	Japan
City	Yokohama
Period	26/04/25 → 1/05/25
Internet address	https://chi2025.acm.org/

Funding

This research was partially supported by the National Natural Science Foundation of China, Young Scientists Fund (Project No. 62402301) and the STU Scientific Research Initiation Grant (SRIG, Project No. NTF23024).

Research Keywords

Acoustic Sensing
Text Input
Touchscreen Interaction

Access Output

10.1145/3706599.3720116

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Liu, Y., Xiao, D., Chen, T., & Zhu, K. (2025). Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input. In N. Yamashita, V. Evers, K. Yatani, & X. Ding (Eds.), CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems Article 87 (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3706599.3720116

Liu, Yantao ; Xiao, Dongmin ; Chen, Taizhou et al. / Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input. CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. editor / Naomi Yamashita ; Vanessa Evers ; Koji Yatani ; Xianghua (Sharon) Ding. Association for Computing Machinery, 2025. (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{6502154b804d4466b39d5976e448e418,

title = "Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input",

abstract = "Touchscreen typing on tablet has become popular in modern digital routine, calling for investigation of more expressive input method on touchscreen keyboard. In this paper, we propose a novel approach to augment touchscreen typing experience by integrating key-press finger posture recognition to extend the input space of a standard touchscreen keyboard. Our system distinguish between finger tip contact(FP) and finger pad contact(FP) through acoustic sensing, enabling seamless switching between normal and functional input. We evaluate the performance of our system through offline and online experiments, where we show that our system achieves an offline key-wise recognition accuracy of up to 96.3%. The online experiment shows a real-time recognition accuracy of 94% and 88% in quiet and noisy environments, respectively. We further conducted a usability study on text formatting task, which shows that our method significantly outperform the baseline method in terms of input speed and functionality. {\textcopyright} 2025 Copyright held by the owner/author(s).",

keywords = "Acoustic Sensing, Text Input, Touchscreen Interaction",

author = "Yantao Liu and Dongmin Xiao and Taizhou Chen and Kening Zhu",

year = "2025",

month = apr,

doi = "10.1145/3706599.3720116",

language = "English",

isbn = "979-8-4007-1395-8",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

editor = "Naomi Yamashita and Vanessa Evers and Koji Yatani and Ding, {Xianghua (Sharon)}",

booktitle = "CHI EA '25",

address = "United States",

note = "2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025) : IkiCHI, CHI'25 ; Conference date: 26-04-2025 Through 01-05-2025",

url = "https://chi2025.acm.org/",

}

Liu, Y, Xiao, D, Chen, T & Zhu, K 2025, Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input. in N Yamashita, V Evers, K Yatani & X Ding (eds), CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems., 87, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025), Yokohama, Japan, 26/04/25. https://doi.org/10.1145/3706599.3720116

Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input. / Liu, Yantao; Xiao, Dongmin; Chen, Taizhou et al.
CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. ed. / Naomi Yamashita; Vanessa Evers; Koji Yatani; Xianghua (Sharon) Ding. Association for Computing Machinery, 2025. 87 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input

AU - Liu, Yantao

AU - Xiao, Dongmin

AU - Chen, Taizhou

AU - Zhu, Kening

PY - 2025/4

Y1 - 2025/4

N2 - Touchscreen typing on tablet has become popular in modern digital routine, calling for investigation of more expressive input method on touchscreen keyboard. In this paper, we propose a novel approach to augment touchscreen typing experience by integrating key-press finger posture recognition to extend the input space of a standard touchscreen keyboard. Our system distinguish between finger tip contact(FP) and finger pad contact(FP) through acoustic sensing, enabling seamless switching between normal and functional input. We evaluate the performance of our system through offline and online experiments, where we show that our system achieves an offline key-wise recognition accuracy of up to 96.3%. The online experiment shows a real-time recognition accuracy of 94% and 88% in quiet and noisy environments, respectively. We further conducted a usability study on text formatting task, which shows that our method significantly outperform the baseline method in terms of input speed and functionality. © 2025 Copyright held by the owner/author(s).

AB - Touchscreen typing on tablet has become popular in modern digital routine, calling for investigation of more expressive input method on touchscreen keyboard. In this paper, we propose a novel approach to augment touchscreen typing experience by integrating key-press finger posture recognition to extend the input space of a standard touchscreen keyboard. Our system distinguish between finger tip contact(FP) and finger pad contact(FP) through acoustic sensing, enabling seamless switching between normal and functional input. We evaluate the performance of our system through offline and online experiments, where we show that our system achieves an offline key-wise recognition accuracy of up to 96.3%. The online experiment shows a real-time recognition accuracy of 94% and 88% in quiet and noisy environments, respectively. We further conducted a usability study on text formatting task, which shows that our method significantly outperform the baseline method in terms of input speed and functionality. © 2025 Copyright held by the owner/author(s).

KW - Acoustic Sensing

KW - Text Input

KW - Touchscreen Interaction

UR - http://www.scopus.com/inward/record.url?scp=105005758582&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-105005758582&origin=recordpage

U2 - 10.1145/3706599.3720116

DO - 10.1145/3706599.3720116

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 979-8-4007-1395-8

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI EA '25

A2 - Yamashita, Naomi

A2 - Evers, Vanessa

A2 - Yatani, Koji

A2 - Ding, Xianghua (Sharon)

PB - Association for Computing Machinery

T2 - 2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025)

Y2 - 26 April 2025 through 1 May 2025

ER -

Liu Y, Xiao D, Chen T , Zhu K. Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input. In Yamashita N, Evers V, Yatani K, Ding X, editors, CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2025. 87. (Conference on Human Factors in Computing Systems - Proceedings). Epub 2025 Apr 25. doi: 10.1145/3706599.3720116

Augmenting Tablet Typing Experience by Integrating Key-Press Finger Contact Types as Input

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this