Bidirectional Shadow Rendering for Interactive Mixed 360° Videos

Lili Wang; Hao Wang; Danqing Dai; Jiaye Leng; Xiaoguang Han

doi:10.1109/VR50410.2021.00038

Bidirectional Shadow Rendering for Interactive Mixed 360° Videos

Lili Wang^*
, Hao Wang
, Danqing Dai
, Jiaye Leng
, Xiaoguang Han

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

8 Citations (Scopus)

Abstract

In this paper, we provide a bidirectional shadow rendering method to render shadows between real and virtual objects in the 360° videos in real time. We construct a 3D scene approximation from the current output viewpoint to approximate the real scene geometry nearby in the video. Then, we propose a ray casting based algorithm to determine the shadow regions on the virtual objects cast by the real objects. After that, we introduce an object-aware shadow mapping method to cast shadows from virtual objects to real objects. Finally, we use a shadow intensity estimation algorithm to determine the shadow intensity of virtual objects and real objects to obtain shadows consistent with the input 360° video. The experiment results prove the effectiveness of our bidirectional shadow rendering method for mixed 360° videos. Our method can generate visually realistic shadows for virtual objects and real objects in 360° video in realtime, and make virtual objects more natural to integrate with real scenes in 360° videos of the mixed reality applications. ©2021 IEEE

Original language	English
Title of host publication	Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021)
Publisher	IEEE
Pages	170-178
ISBN (Electronic)	978-0-7381-2556-5
DOIs	https://doi.org/10.1109/VR50410.2021.00038
Publication status	Published - 10 May 2021
Externally published	Yes
Event	28th IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021) - Virtual, Lisboa, Portugal Duration: 27 Mar 2021 → 3 Apr 2021

Conference

Conference	28th IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021)
Abbreviated title	IEEE VR 2021
Place	Portugal
City	Lisboa
Period	27/03/21 → 3/04/21

Research Keywords

Mixed reality-360°
videos-Shadow rendering

Access Output

10.1109/VR50410.2021.00038

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{f9fe38df51e4464288cf5d491efa46f4,

title = "Bidirectional Shadow Rendering for Interactive Mixed 360° Videos",

abstract = "In this paper, we provide a bidirectional shadow rendering method to render shadows between real and virtual objects in the 360° videos in real time. We construct a 3D scene approximation from the current output viewpoint to approximate the real scene geometry nearby in the video. Then, we propose a ray casting based algorithm to determine the shadow regions on the virtual objects cast by the real objects. After that, we introduce an object-aware shadow mapping method to cast shadows from virtual objects to real objects. Finally, we use a shadow intensity estimation algorithm to determine the shadow intensity of virtual objects and real objects to obtain shadows consistent with the input 360° video. The experiment results prove the effectiveness of our bidirectional shadow rendering method for mixed 360° videos. Our method can generate visually realistic shadows for virtual objects and real objects in 360° video in realtime, and make virtual objects more natural to integrate with real scenes in 360° videos of the mixed reality applications. {\textcopyright}2021 IEEE",

keywords = "Mixed reality-360°, videos-Shadow rendering",

author = "Lili Wang and Hao Wang and Danqing Dai and Jiaye Leng and Xiaoguang Han",

year = "2021",

month = may,

day = "10",

doi = "10.1109/VR50410.2021.00038",

language = "English",

pages = "170--178",

booktitle = "Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021)",

publisher = "IEEE",

address = "United States",

note = "28th IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021), IEEE VR 2021 ; Conference date: 27-03-2021 Through 03-04-2021",

}

Wang, L, Wang, H, Dai, D, Leng, J & Han, X 2021, Bidirectional Shadow Rendering for Interactive Mixed 360° Videos. in Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021). IEEE, pp. 170-178, 28th IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021), Lisboa, Portugal, 27/03/21. https://doi.org/10.1109/VR50410.2021.00038

Bidirectional Shadow Rendering for Interactive Mixed 360° Videos. / Wang, Lili; Wang, Hao; Dai, Danqing et al.
Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021). IEEE, 2021. p. 170-178.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Bidirectional Shadow Rendering for Interactive Mixed 360° Videos

AU - Wang, Lili

AU - Wang, Hao

AU - Dai, Danqing

AU - Leng, Jiaye

AU - Han, Xiaoguang

PY - 2021/5/10

Y1 - 2021/5/10

N2 - In this paper, we provide a bidirectional shadow rendering method to render shadows between real and virtual objects in the 360° videos in real time. We construct a 3D scene approximation from the current output viewpoint to approximate the real scene geometry nearby in the video. Then, we propose a ray casting based algorithm to determine the shadow regions on the virtual objects cast by the real objects. After that, we introduce an object-aware shadow mapping method to cast shadows from virtual objects to real objects. Finally, we use a shadow intensity estimation algorithm to determine the shadow intensity of virtual objects and real objects to obtain shadows consistent with the input 360° video. The experiment results prove the effectiveness of our bidirectional shadow rendering method for mixed 360° videos. Our method can generate visually realistic shadows for virtual objects and real objects in 360° video in realtime, and make virtual objects more natural to integrate with real scenes in 360° videos of the mixed reality applications. ©2021 IEEE

AB - In this paper, we provide a bidirectional shadow rendering method to render shadows between real and virtual objects in the 360° videos in real time. We construct a 3D scene approximation from the current output viewpoint to approximate the real scene geometry nearby in the video. Then, we propose a ray casting based algorithm to determine the shadow regions on the virtual objects cast by the real objects. After that, we introduce an object-aware shadow mapping method to cast shadows from virtual objects to real objects. Finally, we use a shadow intensity estimation algorithm to determine the shadow intensity of virtual objects and real objects to obtain shadows consistent with the input 360° video. The experiment results prove the effectiveness of our bidirectional shadow rendering method for mixed 360° videos. Our method can generate visually realistic shadows for virtual objects and real objects in 360° video in realtime, and make virtual objects more natural to integrate with real scenes in 360° videos of the mixed reality applications. ©2021 IEEE

KW - Mixed reality-360°

KW - videos-Shadow rendering

UR - http://www.scopus.com/inward/record.url?scp=85106488187&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85106488187&origin=recordpage

U2 - 10.1109/VR50410.2021.00038

DO - 10.1109/VR50410.2021.00038

M3 - RGC 32 - Refereed conference paper (with host publication)

SP - 170

EP - 178

BT - Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021)

PB - IEEE

T2 - 28th IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2021)

Y2 - 27 March 2021 through 3 April 2021

ER -

Bidirectional Shadow Rendering for Interactive Mixed 360° Videos

Abstract

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this