Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness

Geert Lugtenberg; Wolfgang Hürst; Nina Rosa; Christian Sandor; Alexander Plopski; Takafumi Taketomi; Hirokazu Kato

doi:10.1007/978-3-319-73603-7_30

Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness

Geert Lugtenberg
, Wolfgang Hürst^*
, Nina Rosa
, Christian Sandor
, Alexander Plopski
, Takafumi Taketomi
, Hirokazu Kato

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

3 Citations (Scopus)

Abstract

With vision being a primary sense of humans, we often first estimate the physical properties of objects by looking at them. However, when in doubt, for example, about the material they are made of or its structure, it is natural to apply other senses, such as haptics by touching them. Aiming at the ultimate goal of achieving a full-sensory augmented reality experience, we present an initial study focusing on multimodal feedback when tapping an object to estimate the thickness of its material. Our results indicate that we can change the perception of thickness of stiff objects by modulating acoustic stimuli. For flexible objects, which have a more distinctive tactile characteristic, adding vibratory responses when tapping on thick objects can make people perceive them as thin. We also identified that in the latter case, adding congruent acoustic stimuli does not further enhance the illusion but worsens it.

Original language	English
Title of host publication	MultiMedia Modeling
Subtitle of host publication	24th International Conference, MMM 2018, Proceedings, Part I
Editors	Klaus Schoeffmann, Thanarat H. Chalidabhongse, Wah Ngo Chong, Supavadee Aramvith, Noel E. O’Connor, Yo-Sung Ho, Moncef Gabbouj, Ahmed Elgammal
Publisher	Springer, Cham
Pages	369-380
ISBN (Electronic)	978-3-319-73603-7
ISBN (Print)	978-3-319-73602-0
DOIs	https://doi.org/10.1007/978-3-319-73603-7_30
Publication status	Published - Feb 2018
Externally published	Yes
Event	24th International Conference on MultiMedia Modeling, MMM 2018 - Bangkok, Thailand Duration: 5 Feb 2018 → 7 Feb 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10704 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on MultiMedia Modeling, MMM 2018
Place	Thailand
City	Bangkok
Period	5/02/18 → 7/02/18

Research Keywords

Augmented reality
Multimodal AR
Multimodal perception

Access Output

10.1007/978-3-319-73603-7_30

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Lugtenberg, G., Hürst, W., Rosa, N., Sandor, C., Plopski, A., Taketomi, T., & Kato, H. (2018). Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. In K. Schoeffmann, T. H. Chalidabhongse, W. N. Chong, S. Aramvith, N. E. O’Connor, Y.-S. Ho, M. Gabbouj, & A. Elgammal (Eds.), MultiMedia Modeling: 24th International Conference, MMM 2018, Proceedings, Part I (pp. 369-380). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10704 LNCS). Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_30

Lugtenberg, Geert ; Hürst, Wolfgang ; Rosa, Nina et al. / Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. MultiMedia Modeling: 24th International Conference, MMM 2018, Proceedings, Part I. editor / Klaus Schoeffmann ; Thanarat H. Chalidabhongse ; Wah Ngo Chong ; Supavadee Aramvith ; Noel E. O’Connor ; Yo-Sung Ho ; Moncef Gabbouj ; Ahmed Elgammal. Springer, Cham, 2018. pp. 369-380 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b19443b9e45c482088e1e2c138cf721a,

title = "Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness",

abstract = "With vision being a primary sense of humans, we often first estimate the physical properties of objects by looking at them. However, when in doubt, for example, about the material they are made of or its structure, it is natural to apply other senses, such as haptics by touching them. Aiming at the ultimate goal of achieving a full-sensory augmented reality experience, we present an initial study focusing on multimodal feedback when tapping an object to estimate the thickness of its material. Our results indicate that we can change the perception of thickness of stiff objects by modulating acoustic stimuli. For flexible objects, which have a more distinctive tactile characteristic, adding vibratory responses when tapping on thick objects can make people perceive them as thin. We also identified that in the latter case, adding congruent acoustic stimuli does not further enhance the illusion but worsens it.",

keywords = "Augmented reality, Multimodal AR, Multimodal perception",

author = "Geert Lugtenberg and Wolfgang H{\"u}rst and Nina Rosa and Christian Sandor and Alexander Plopski and Takafumi Taketomi and Hirokazu Kato",

year = "2018",

month = feb,

doi = "10.1007/978-3-319-73603-7\_30",

language = "English",

isbn = "978-3-319-73602-0",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Cham",

pages = "369--380",

editor = "Klaus Schoeffmann and \{H. Chalidabhongse\}, \{Thanarat \} and Chong, \{Wah Ngo\} and Aramvith, \{Supavadee \} and O{\textquoteright}Connor, \{Noel E. \} and Yo-Sung Ho and Gabbouj, \{Moncef \} and Elgammal, \{Ahmed \}",

booktitle = "MultiMedia Modeling",

note = "24th International Conference on MultiMedia Modeling, MMM 2018 ; Conference date: 05-02-2018 Through 07-02-2018",

}

Lugtenberg, G, Hürst, W, Rosa, N, Sandor, C, Plopski, A, Taketomi, T & Kato, H 2018, Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. in K Schoeffmann, T H. Chalidabhongse, WN Chong, S Aramvith, NE O’Connor, Y-S Ho, M Gabbouj & A Elgammal (eds), MultiMedia Modeling: 24th International Conference, MMM 2018, Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10704 LNCS, Springer, Cham, pp. 369-380, 24th International Conference on MultiMedia Modeling, MMM 2018, Bangkok, Thailand, 5/02/18. https://doi.org/10.1007/978-3-319-73603-7_30

Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. / Lugtenberg, Geert; Hürst, Wolfgang; Rosa, Nina et al.
MultiMedia Modeling: 24th International Conference, MMM 2018, Proceedings, Part I. ed. / Klaus Schoeffmann; Thanarat H. Chalidabhongse; Wah Ngo Chong; Supavadee Aramvith; Noel E. O’Connor; Yo-Sung Ho; Moncef Gabbouj; Ahmed Elgammal. Springer, Cham, 2018. p. 369-380 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10704 LNCS).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness

AU - Lugtenberg, Geert

AU - Hürst, Wolfgang

AU - Rosa, Nina

AU - Sandor, Christian

AU - Plopski, Alexander

AU - Taketomi, Takafumi

AU - Kato, Hirokazu

PY - 2018/2

Y1 - 2018/2

N2 - With vision being a primary sense of humans, we often first estimate the physical properties of objects by looking at them. However, when in doubt, for example, about the material they are made of or its structure, it is natural to apply other senses, such as haptics by touching them. Aiming at the ultimate goal of achieving a full-sensory augmented reality experience, we present an initial study focusing on multimodal feedback when tapping an object to estimate the thickness of its material. Our results indicate that we can change the perception of thickness of stiff objects by modulating acoustic stimuli. For flexible objects, which have a more distinctive tactile characteristic, adding vibratory responses when tapping on thick objects can make people perceive them as thin. We also identified that in the latter case, adding congruent acoustic stimuli does not further enhance the illusion but worsens it.

AB - With vision being a primary sense of humans, we often first estimate the physical properties of objects by looking at them. However, when in doubt, for example, about the material they are made of or its structure, it is natural to apply other senses, such as haptics by touching them. Aiming at the ultimate goal of achieving a full-sensory augmented reality experience, we present an initial study focusing on multimodal feedback when tapping an object to estimate the thickness of its material. Our results indicate that we can change the perception of thickness of stiff objects by modulating acoustic stimuli. For flexible objects, which have a more distinctive tactile characteristic, adding vibratory responses when tapping on thick objects can make people perceive them as thin. We also identified that in the latter case, adding congruent acoustic stimuli does not further enhance the illusion but worsens it.

KW - Augmented reality

KW - Multimodal AR

KW - Multimodal perception

UR - https://www.scopus.com/pages/publications/85042104957

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85042104957&origin=recordpage

U2 - 10.1007/978-3-319-73603-7_30

DO - 10.1007/978-3-319-73603-7_30

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-3-319-73602-0

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 369

EP - 380

BT - MultiMedia Modeling

A2 - Schoeffmann, Klaus

A2 - H. Chalidabhongse, Thanarat

A2 - Chong, Wah Ngo

A2 - Aramvith, Supavadee

A2 - O’Connor, Noel E.

A2 - Ho, Yo-Sung

A2 - Gabbouj, Moncef

A2 - Elgammal, Ahmed

PB - Springer, Cham

T2 - 24th International Conference on MultiMedia Modeling, MMM 2018

Y2 - 5 February 2018 through 7 February 2018

ER -

Lugtenberg G, Hürst W, Rosa N, Sandor C, Plopski A, Taketomi T et al. Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness. In Schoeffmann K, H. Chalidabhongse T, Chong WN, Aramvith S, O’Connor NE, Ho YS, Gabbouj M, Elgammal A, editors, MultiMedia Modeling: 24th International Conference, MMM 2018, Proceedings, Part I. Springer, Cham. 2018. p. 369-380. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-73603-7_30

Multimodal Augmented Reality - Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this