AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks

Felicia Fang-Yi Tan; Peisen Xu; Ashwin Ram; Wei Zhen Suen; Shengdong Zhao; Yun Huang; Christophe Hurter

doi:10.1145/3613904.3642514

AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks

Felicia Fang-Yi Tan, Peisen Xu, Ashwin Ram, Wei Zhen Suen, Shengdong Zhao^*, Yun Huang, Christophe Hurter

^*Corresponding author for this work

City University of Hong Kong

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

8 Citations (Scopus)

Abstract

The rise of multitasking in contemporary lifestyles has positioned audio-first content as an essential medium for information consumption. We present AudioXtend, an approach to augment audiobook experiences during daily tasks by integrating glanceable, AI-generated visuals through optical see-through head-mounted displays (OHMDs). Our initial study showed that these visual augmentations not only preserved users' primary task efficiency but also dramatically enhanced immediate auditory content recall by 33.3% and 7-day recall by 32.7%, alongside a marked improvement in narrative engagement. Through participatory design workshops involving digital arts designers, we crafted a set of design principles for visual augmentations that are attuned to the requirements of multitaskers. Finally, a 3-day take-home field study further revealed new insights for everyday use, underscoring the potential of assisted reality (aR) to enhance heads-up listening and incidental learning experiences. © 2024 Copyright held by the owner/author(s)

Original language	English
Title of host publication	CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems
Editors	Florian Floyd Mueller, Penny Kyburz
Publisher	Association for Computing Machinery
ISBN (Print)	9798400703300
DOIs	https://doi.org/10.1145/3613904.3642514
Publication status	Published - May 2024
Event	2024 CHI Conference on Human Factors in Computing Sytems (CHI 2024): "Surfing the World" - Hawaiʻi Convention Center, Honolulu, United States Duration: 11 May 2024 → 16 May 2024 https://chi2024.acm.org/for-attendees/

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2024 CHI Conference on Human Factors in Computing Sytems (CHI 2024)
Place	United States
City	Honolulu
Period	11/05/24 → 16/05/24
Internet address	https://chi2024.acm.org/for-attendees/

Research Keywords

Assisted Reality
Audiobook Augmentation
Heads-Up Computing
Incidental learning
Optical See-Through Head-Mounted Displays
Recall Enhancement
Smart-glasses
Visual Storytelling

Access Output

10.1145/3613904.3642514

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Tan, F. F.-Y., Xu, P., Ram, A., Suen, W. Z., Zhao, S., Huang, Y., & Hurter, C. (2024). AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. In F. F. Mueller, & P. Kyburz (Eds.), CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems Article 83 (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3613904.3642514

Tan, Felicia Fang-Yi ; Xu, Peisen ; Ram, Ashwin et al. / AudioXtend : Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems. editor / Florian Floyd Mueller ; Penny Kyburz. Association for Computing Machinery, 2024. (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{c7a1ab57380249c28c619476908601b4,

title = "AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks",

abstract = "The rise of multitasking in contemporary lifestyles has positioned audio-first content as an essential medium for information consumption. We present AudioXtend, an approach to augment audiobook experiences during daily tasks by integrating glanceable, AI-generated visuals through optical see-through head-mounted displays (OHMDs). Our initial study showed that these visual augmentations not only preserved users' primary task efficiency but also dramatically enhanced immediate auditory content recall by 33.3% and 7-day recall by 32.7%, alongside a marked improvement in narrative engagement. Through participatory design workshops involving digital arts designers, we crafted a set of design principles for visual augmentations that are attuned to the requirements of multitaskers. Finally, a 3-day take-home field study further revealed new insights for everyday use, underscoring the potential of assisted reality (aR) to enhance heads-up listening and incidental learning experiences. {\textcopyright} 2024 Copyright held by the owner/author(s)",

keywords = "Assisted Reality, Audiobook Augmentation, Heads-Up Computing, Incidental learning, Optical See-Through Head-Mounted Displays, Recall Enhancement, Smart-glasses, Visual Storytelling",

author = "Tan, {Felicia Fang-Yi} and Peisen Xu and Ashwin Ram and Suen, {Wei Zhen} and Shengdong Zhao and Yun Huang and Christophe Hurter",

year = "2024",

month = may,

doi = "10.1145/3613904.3642514",

language = "English",

isbn = "9798400703300",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

editor = "Mueller, {Florian Floyd} and Penny Kyburz",

booktitle = "CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems",

address = "United States",

note = "2024 CHI Conference on Human Factors in Computing Sytems (CHI 2024) : {"}Surfing the World{"} ; Conference date: 11-05-2024 Through 16-05-2024",

url = "https://chi2024.acm.org/for-attendees/",

}

Tan, FF-Y, Xu, P, Ram, A, Suen, WZ, Zhao, S, Huang, Y & Hurter, C 2024, AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. in FF Mueller & P Kyburz (eds), CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems., 83, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2024 CHI Conference on Human Factors in Computing Sytems (CHI 2024), Honolulu, United States, 11/05/24. https://doi.org/10.1145/3613904.3642514

AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. / Tan, Felicia Fang-Yi; Xu, Peisen; Ram, Ashwin et al.
CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems. ed. / Florian Floyd Mueller; Penny Kyburz. Association for Computing Machinery, 2024. 83 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - AudioXtend

T2 - 2024 CHI Conference on Human Factors in Computing Sytems (CHI 2024)

AU - Tan, Felicia Fang-Yi

AU - Xu, Peisen

AU - Ram, Ashwin

AU - Suen, Wei Zhen

AU - Zhao, Shengdong

AU - Huang, Yun

AU - Hurter, Christophe

PY - 2024/5

Y1 - 2024/5

N2 - The rise of multitasking in contemporary lifestyles has positioned audio-first content as an essential medium for information consumption. We present AudioXtend, an approach to augment audiobook experiences during daily tasks by integrating glanceable, AI-generated visuals through optical see-through head-mounted displays (OHMDs). Our initial study showed that these visual augmentations not only preserved users' primary task efficiency but also dramatically enhanced immediate auditory content recall by 33.3% and 7-day recall by 32.7%, alongside a marked improvement in narrative engagement. Through participatory design workshops involving digital arts designers, we crafted a set of design principles for visual augmentations that are attuned to the requirements of multitaskers. Finally, a 3-day take-home field study further revealed new insights for everyday use, underscoring the potential of assisted reality (aR) to enhance heads-up listening and incidental learning experiences. © 2024 Copyright held by the owner/author(s)

AB - The rise of multitasking in contemporary lifestyles has positioned audio-first content as an essential medium for information consumption. We present AudioXtend, an approach to augment audiobook experiences during daily tasks by integrating glanceable, AI-generated visuals through optical see-through head-mounted displays (OHMDs). Our initial study showed that these visual augmentations not only preserved users' primary task efficiency but also dramatically enhanced immediate auditory content recall by 33.3% and 7-day recall by 32.7%, alongside a marked improvement in narrative engagement. Through participatory design workshops involving digital arts designers, we crafted a set of design principles for visual augmentations that are attuned to the requirements of multitaskers. Finally, a 3-day take-home field study further revealed new insights for everyday use, underscoring the potential of assisted reality (aR) to enhance heads-up listening and incidental learning experiences. © 2024 Copyright held by the owner/author(s)

KW - Assisted Reality

KW - Audiobook Augmentation

KW - Heads-Up Computing

KW - Incidental learning

KW - Optical See-Through Head-Mounted Displays

KW - Recall Enhancement

KW - Smart-glasses

KW - Visual Storytelling

UR - http://www.scopus.com/inward/record.url?scp=85194814965&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85194814965&origin=recordpage

U2 - 10.1145/3613904.3642514

DO - 10.1145/3613904.3642514

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9798400703300

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems

A2 - Mueller, Florian Floyd

A2 - Kyburz, Penny

PB - Association for Computing Machinery

Y2 - 11 May 2024 through 16 May 2024

ER -

Tan FFY, Xu P, Ram A, Suen WZ, Zhao S, Huang Y et al. AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. In Mueller FF, Kyburz P, editors, CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2024. 83. (Conference on Human Factors in Computing Systems - Proceedings). Epub 2024 May 11. doi: 10.1145/3613904.3642514

AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this