FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams

Arshad Nasser; Taizhou Chen; Can Liu; Kening Zhu; P V M Rao

doi:10.1007/978-3-030-49062-1_32

FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams

Arshad Nasser^*
, Taizhou Chen
, Can Liu
, Kening Zhu
, P V M Rao

^*Corresponding author for this work

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

4 Citations (Scopus)

Abstract

Traditional tactile diagrams for the visually-impaired (VI) use short Braille keys and annotations to provide additional information in separate Braille legend pages. Frequent navigation between the tactile diagram and the annex pages during the diagram exploration results in low efficiency in diagram comprehension. We present the design of FingerTalkie, a finger-worn device that uses discrete colors on a color-tagged tactile diagram for interactive audio labeling of the graphical elements. Through an iterative design process involving 8 VI users, we designed a unique offset point-and-click technique that enables the bimanual exploration of the diagrams without hindering the tactile perception of the fingertips. Unlike existing camera-based and finger-worn audio-tactile devices, FingerTalkie supports one-finger interaction and can work in any lighting conditions without calibration. We conducted a controlled experiment with 12 blind-folded sighted users to evaluate the usability of the device. Further, a focus-group interview with 8 VI users shows their appreciation for the FingerTalkie’s ease of use, support for two-hand exploration, and its potential in improving the efficiency of comprehending tactile diagrams by replacing Braille labels.

Original language	English
Title of host publication	Human-Computer Interaction
Subtitle of host publication	Multimodal and Natural Interaction
Editors	Masaaki Kurosu
Publisher	Springer
Pages	475-496
ISBN (Electronic)	9783030490621
ISBN (Print)	9783030490614
DOIs	https://doi.org/10.1007/978-3-030-49062-1_32
Publication status	Published - Jul 2020
Event	Thematic Area, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 - AC Bella Sky Hotel and Bella Center (Virtual), Copenhagen, Denmark Duration: 19 Jul 2020 → 24 Jul 2020 http://2020.hci.international/

Publication series

Name	Lecture Notes in Computer Science
Volume	12182
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Thematic Area, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020
Abbreviated title	HCII 2020
Place	Denmark
City	Copenhagen
Period	19/07/20 → 24/07/20
Internet address	http://2020.hci.international/

Research Keywords

Audio-tactile diagram
Finger-worn device
Offset point and click
Blind
Visually impaired

Access Output

10.1007/978-3-030-49062-1_32

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Nasser, A., Chen, T., Liu, C., Zhu, K., & Rao, P. V. M. (2020). FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams. In M. Kurosu (Ed.), Human-Computer Interaction: Multimodal and Natural Interaction (pp. 475-496). (Lecture Notes in Computer Science; Vol. 12182). Springer . https://doi.org/10.1007/978-3-030-49062-1_32

@inproceedings{5c4afef611b34051a9760459c373557a,

title = "FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams",

abstract = "Traditional tactile diagrams for the visually-impaired (VI) use short Braille keys and annotations to provide additional information in separate Braille legend pages. Frequent navigation between the tactile diagram and the annex pages during the diagram exploration results in low efficiency in diagram comprehension. We present the design of FingerTalkie, a finger-worn device that uses discrete colors on a color-tagged tactile diagram for interactive audio labeling of the graphical elements. Through an iterative design process involving 8 VI users, we designed a unique offset point-and-click technique that enables the bimanual exploration of the diagrams without hindering the tactile perception of the fingertips. Unlike existing camera-based and finger-worn audio-tactile devices, FingerTalkie supports one-finger interaction and can work in any lighting conditions without calibration. We conducted a controlled experiment with 12 blind-folded sighted users to evaluate the usability of the device. Further, a focus-group interview with 8 VI users shows their appreciation for the FingerTalkie{\textquoteright}s ease of use, support for two-hand exploration, and its potential in improving the efficiency of comprehending tactile diagrams by replacing Braille labels.",

keywords = "Audio-tactile diagram, Finger-worn device, Offset point and click, Blind, Visually impaired",

author = "Arshad Nasser and Taizhou Chen and Can Liu and Kening Zhu and Rao, \{P V M\}",

year = "2020",

month = jul,

doi = "10.1007/978-3-030-49062-1\_32",

language = "English",

isbn = "9783030490614",

series = "Lecture Notes in Computer Science",

publisher = "Springer ",

pages = "475--496",

editor = "Masaaki Kurosu",

booktitle = "Human-Computer Interaction",

note = "Thematic Area, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, HCII 2020 ; Conference date: 19-07-2020 Through 24-07-2020",

url = "http://2020.hci.international/",

}

Nasser, A , Chen, T , Liu, C , Zhu, K & Rao, PVM 2020, FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams. in M Kurosu (ed.), Human-Computer Interaction: Multimodal and Natural Interaction. Lecture Notes in Computer Science, vol. 12182, Springer , pp. 475-496, Thematic Area, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, Copenhagen, Denmark, 19/07/20. https://doi.org/10.1007/978-3-030-49062-1_32

FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams. / Nasser, Arshad ; Chen, Taizhou ; Liu, Can et al.
Human-Computer Interaction: Multimodal and Natural Interaction. ed. / Masaaki Kurosu. Springer , 2020. p. 475-496 (Lecture Notes in Computer Science; Vol. 12182).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - FingerTalkie

T2 - Thematic Area, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020

AU - Nasser, Arshad

AU - Chen, Taizhou

AU - Liu, Can

AU - Zhu, Kening

AU - Rao, P V M

PY - 2020/7

Y1 - 2020/7

N2 - Traditional tactile diagrams for the visually-impaired (VI) use short Braille keys and annotations to provide additional information in separate Braille legend pages. Frequent navigation between the tactile diagram and the annex pages during the diagram exploration results in low efficiency in diagram comprehension. We present the design of FingerTalkie, a finger-worn device that uses discrete colors on a color-tagged tactile diagram for interactive audio labeling of the graphical elements. Through an iterative design process involving 8 VI users, we designed a unique offset point-and-click technique that enables the bimanual exploration of the diagrams without hindering the tactile perception of the fingertips. Unlike existing camera-based and finger-worn audio-tactile devices, FingerTalkie supports one-finger interaction and can work in any lighting conditions without calibration. We conducted a controlled experiment with 12 blind-folded sighted users to evaluate the usability of the device. Further, a focus-group interview with 8 VI users shows their appreciation for the FingerTalkie’s ease of use, support for two-hand exploration, and its potential in improving the efficiency of comprehending tactile diagrams by replacing Braille labels.

AB - Traditional tactile diagrams for the visually-impaired (VI) use short Braille keys and annotations to provide additional information in separate Braille legend pages. Frequent navigation between the tactile diagram and the annex pages during the diagram exploration results in low efficiency in diagram comprehension. We present the design of FingerTalkie, a finger-worn device that uses discrete colors on a color-tagged tactile diagram for interactive audio labeling of the graphical elements. Through an iterative design process involving 8 VI users, we designed a unique offset point-and-click technique that enables the bimanual exploration of the diagrams without hindering the tactile perception of the fingertips. Unlike existing camera-based and finger-worn audio-tactile devices, FingerTalkie supports one-finger interaction and can work in any lighting conditions without calibration. We conducted a controlled experiment with 12 blind-folded sighted users to evaluate the usability of the device. Further, a focus-group interview with 8 VI users shows their appreciation for the FingerTalkie’s ease of use, support for two-hand exploration, and its potential in improving the efficiency of comprehending tactile diagrams by replacing Braille labels.

KW - Audio-tactile diagram

KW - Finger-worn device

KW - Offset point and click

KW - Blind

KW - Visually impaired

UR - https://www.scopus.com/pages/publications/85088752109

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85088752109&origin=recordpage

U2 - 10.1007/978-3-030-49062-1_32

DO - 10.1007/978-3-030-49062-1_32

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9783030490614

T3 - Lecture Notes in Computer Science

SP - 475

EP - 496

BT - Human-Computer Interaction

A2 - Kurosu, Masaaki

PB - Springer

Y2 - 19 July 2020 through 24 July 2020

ER -

FingerTalkie: Designing a Low-Cost Finger-Worn Device for Interactive Audio Labeling of Tactile Diagrams

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this