Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems

Lingyan Ruan; Bin Chen; Chi-Hang Chow; Hiu-Lok Chung; Pok-Yin Victor Leung; Miu-Ling Lam

doi:10.1145/3757376.3771419

Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems

Lingyan Ruan
, Bin Chen
, Chi-Hang Chow
, Hiu-Lok Chung
, Pok-Yin Victor Leung
, Miu-Ling Lam^*

^*Corresponding author for this work

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Immersive panoramic 3D visualization systems enhance presence, enabling rich educational, cultural, and collaborative experiences. While gaze offers significant potential as a natural, hands-free interaction modality in such environments, existing solutions often depend on fixed camera setups, fiducial markers, or controller-based navigation, which limit user mobility, reduce gaze estimation accuracy, and introduce visual or physical distractions. We present a real-time, multi-user gaze-driven interaction system for panoramic 3D visualization. The system features a customized unit integrating binocular eye tracking, active stereoscopic glasses, and a scene camera, paired with a portable Android device. The scene camera enables precise gaze mapping onto panoramic 3D content through feature-based alignment with the 360° footage, while the Android device handles real-time gaze estimation and synchronization to ensure low-latency interaction. This approach delivers accurate, responsive gaze mapping for multiple freely moving users without the need for additional tracking infrastructure. Two interactive museum applications demonstrate the system’s capability to support fluid, collaborative, and unobtrusive gaze-based interaction in dynamic cultural and educational installations.
© 2025 Copyright held by the owner/author(s).

Original language	English
Title of host publication	SA Technical Communications '25
Subtitle of host publication	Proceedings of the SIGGRAPH Asia 2025 Technical Communications
Editors	Taku Komura, Yuting Ye
Publisher	Association for Computing Machinery
Number of pages	4
ISBN (Print)	979-8-4007-2136-6
DOIs	https://doi.org/10.1145/3757376.3771419
Publication status	Published - 14 Dec 2025
Event	18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2025) - Hong Kong Convention and Exhibition Centre (HKCEC), Hong Kong, China Duration: 15 Dec 2025 → 18 Dec 2025 https://asia.siggraph.org/2025/

Conference

Conference	18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2025)
Abbreviated title	SA '25
Place	Hong Kong, China
Period	15/12/25 → 18/12/25
Internet address	https://asia.siggraph.org/2025/

Bibliographical note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Funding

This work was supported in part by a grant from the Innovation and Technology Commission of the Hong Kong SAR, China (ITS/326/21FP).

Access Output

10.1145/3757376.3771419

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

ITF-ExtU-Lead: Future Cinema Systems: Next-Generation Art Technologies
Shaw, J. (Main Project Coordinator [External]), ALLEN, R. W. (Principal Investigator / Project Coordinator), CASSINELLI, A. (Co-Investigator), CHAN, H. M. (Co-Investigator), LAM, M. L. (Co-Investigator), LIU, C. (Co-Investigator) & ZHU, K. (Co-Investigator)
1/11/22 → …
Project: Research

Cite this

Ruan, L., Chen, B., Chow, C.-H., Chung, H.-L., Leung, P.-Y. V., & Lam, M.-L. (2025). Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems. In T. Komura, & Y. Ye (Eds.), SA Technical Communications '25: Proceedings of the SIGGRAPH Asia 2025 Technical Communications Article 35 Association for Computing Machinery. https://doi.org/10.1145/3757376.3771419

@inproceedings{15b7a9a5b6674d97b5bd54993eab8da3,

title = "Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems",

abstract = "Immersive panoramic 3D visualization systems enhance presence, enabling rich educational, cultural, and collaborative experiences. While gaze offers significant potential as a natural, hands-free interaction modality in such environments, existing solutions often depend on fixed camera setups, fiducial markers, or controller-based navigation, which limit user mobility, reduce gaze estimation accuracy, and introduce visual or physical distractions. We present a real-time, multi-user gaze-driven interaction system for panoramic 3D visualization. The system features a customized unit integrating binocular eye tracking, active stereoscopic glasses, and a scene camera, paired with a portable Android device. The scene camera enables precise gaze mapping onto panoramic 3D content through feature-based alignment with the 360° footage, while the Android device handles real-time gaze estimation and synchronization to ensure low-latency interaction. This approach delivers accurate, responsive gaze mapping for multiple freely moving users without the need for additional tracking infrastructure. Two interactive museum applications demonstrate the system{\textquoteright}s capability to support fluid, collaborative, and unobtrusive gaze-based interaction in dynamic cultural and educational installations. {\textcopyright} 2025 Copyright held by the owner/author(s). ",

author = "Lingyan Ruan and Bin Chen and Chi-Hang Chow and Hiu-Lok Chung and Leung, \{Pok-Yin Victor\} and Miu-Ling Lam",

note = "Research Unit(s) information for this publication is provided by the author(s) concerned.; 18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2025), SA '25 ; Conference date: 15-12-2025 Through 18-12-2025",

year = "2025",

month = dec,

day = "14",

doi = "10.1145/3757376.3771419",

language = "English",

isbn = "979-8-4007-2136-6",

editor = "Taku Komura and Yuting Ye",

booktitle = "SA Technical Communications '25",

publisher = "Association for Computing Machinery",

address = "United States",

url = "https://asia.siggraph.org/2025/",

}

Ruan, L, Chen, B, Chow, C-H, Chung, H-L, Leung, P-YV & Lam, M-L 2025, Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems. in T Komura & Y Ye (eds), SA Technical Communications '25: Proceedings of the SIGGRAPH Asia 2025 Technical Communications., 35, Association for Computing Machinery, 18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2025), Hong Kong, China, 15/12/25. https://doi.org/10.1145/3757376.3771419

Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems. / Ruan, Lingyan; Chen, Bin; Chow, Chi-Hang et al.
SA Technical Communications '25: Proceedings of the SIGGRAPH Asia 2025 Technical Communications. ed. / Taku Komura; Yuting Ye. Association for Computing Machinery, 2025. 35.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems

AU - Ruan, Lingyan

AU - Chen, Bin

AU - Chow, Chi-Hang

AU - Chung, Hiu-Lok

AU - Leung, Pok-Yin Victor

AU - Lam, Miu-Ling

N1 - Research Unit(s) information for this publication is provided by the author(s) concerned.

PY - 2025/12/14

Y1 - 2025/12/14

N2 - Immersive panoramic 3D visualization systems enhance presence, enabling rich educational, cultural, and collaborative experiences. While gaze offers significant potential as a natural, hands-free interaction modality in such environments, existing solutions often depend on fixed camera setups, fiducial markers, or controller-based navigation, which limit user mobility, reduce gaze estimation accuracy, and introduce visual or physical distractions. We present a real-time, multi-user gaze-driven interaction system for panoramic 3D visualization. The system features a customized unit integrating binocular eye tracking, active stereoscopic glasses, and a scene camera, paired with a portable Android device. The scene camera enables precise gaze mapping onto panoramic 3D content through feature-based alignment with the 360° footage, while the Android device handles real-time gaze estimation and synchronization to ensure low-latency interaction. This approach delivers accurate, responsive gaze mapping for multiple freely moving users without the need for additional tracking infrastructure. Two interactive museum applications demonstrate the system’s capability to support fluid, collaborative, and unobtrusive gaze-based interaction in dynamic cultural and educational installations. © 2025 Copyright held by the owner/author(s).

AB - Immersive panoramic 3D visualization systems enhance presence, enabling rich educational, cultural, and collaborative experiences. While gaze offers significant potential as a natural, hands-free interaction modality in such environments, existing solutions often depend on fixed camera setups, fiducial markers, or controller-based navigation, which limit user mobility, reduce gaze estimation accuracy, and introduce visual or physical distractions. We present a real-time, multi-user gaze-driven interaction system for panoramic 3D visualization. The system features a customized unit integrating binocular eye tracking, active stereoscopic glasses, and a scene camera, paired with a portable Android device. The scene camera enables precise gaze mapping onto panoramic 3D content through feature-based alignment with the 360° footage, while the Android device handles real-time gaze estimation and synchronization to ensure low-latency interaction. This approach delivers accurate, responsive gaze mapping for multiple freely moving users without the need for additional tracking infrastructure. Two interactive museum applications demonstrate the system’s capability to support fluid, collaborative, and unobtrusive gaze-based interaction in dynamic cultural and educational installations. © 2025 Copyright held by the owner/author(s).

U2 - 10.1145/3757376.3771419

DO - 10.1145/3757376.3771419

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 979-8-4007-2136-6

BT - SA Technical Communications '25

A2 - Komura, Taku

A2 - Ye, Yuting

PB - Association for Computing Machinery

T2 - 18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2025)

Y2 - 15 December 2025 through 18 December 2025

ER -

Multi-user Gaze Tracking via Dynamic Image Mapping in 360 Immersive 3D Visualization Systems

Abstract

Conference

Bibliographical note

Funding

Access Output

Other Access Options

Permanent Link

Fingerprint

Projects

ITF-ExtU-Lead: Future Cinema Systems: Next-Generation Art Technologies

Cite this