Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration

Lianxin Xie; Bingbing Zheng; Wen Xue; Yunfei Zhang; Le Jiang; Ruotao Xu; Si Wu; Hau-San Wong

doi:10.1609/aaai.v39i8.32944

Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration

Lianxin Xie
, Bingbing Zheng
, Wen Xue
, Yunfei Zhang
, Le Jiang
, Ruotao Xu
, Si Wu^*
, Hau-San Wong^*

^*Corresponding author for this work

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration. Copyright © 2025, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Original language	English
Title of host publication	Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence
Editors	Toby Walsh, Julie Shah, Zico Kolter
Publisher	AAAI Press
Pages	8736-8744
ISBN (Print)	157735897, 9781577358978
DOIs	https://doi.org/10.1609/aaai.v39i8.32944
Publication status	Published - 2025
Event	39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025) - Pennsylvania Convention Center , Philadelphia, United States Duration: 25 Feb 2025 → 4 Mar 2025 https://aaai.org/conference/aaai/aaai-25/

Publication series

Name	Proceedings of the AAAI Conference on Artificial Intelligence
Number	8
Volume	39
ISSN (Print)	2159-5399
ISSN (Electronic)	2374-3468

Conference

Conference	39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025)
Abbreviated title	AAAI-25
Place	United States
City	Philadelphia
Period	25/02/25 → 4/03/25
Internet address	https://aaai.org/conference/aaai/aaai-25/

Funding

This work was supported in part by the National Natural Science Foundation of China (Project No. 62072189), in part by the GuangDong Basic and Applied Basic Research Foundation (Project No. 2024A1515011437), and in part by TCL Science and Technology Innovation Fund (Project No. 20231752).

Access Output

10.1609/aaai.v39i8.32944

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Xie, L., Zheng, B., Xue, W., Zhang, Y., Jiang, L., Xu, R., Wu, S., & Wong, H.-S. (2025). Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration. In T. Walsh, J. Shah, & Z. Kolter (Eds.), Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence (pp. 8736-8744). (Proceedings of the AAAI Conference on Artificial Intelligence; Vol. 39, No. 8). AAAI Press. https://doi.org/10.1609/aaai.v39i8.32944

@inproceedings{b0efa029cd334c50a9a86749fa8cd594,

title = "Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration",

abstract = "Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration. Copyright {\textcopyright} 2025, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.",

author = "Lianxin Xie and Bingbing Zheng and Wen Xue and Yunfei Zhang and Le Jiang and Ruotao Xu and Si Wu and Hau-San Wong",

year = "2025",

doi = "10.1609/aaai.v39i8.32944",

language = "English",

isbn = "157735897",

series = "Proceedings of the AAAI Conference on Artificial Intelligence",

publisher = "AAAI Press",

number = "8",

pages = "8736--8744",

editor = "Toby Walsh and Julie Shah and Zico Kolter",

booktitle = "Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence",

note = "39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025), AAAI-25 ; Conference date: 25-02-2025 Through 04-03-2025",

url = "https://aaai.org/conference/aaai/aaai-25/",

}

Xie, L, Zheng, B, Xue, W, Zhang, Y, Jiang, L, Xu, R, Wu, S & Wong, H-S 2025, Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration. in T Walsh, J Shah & Z Kolter (eds), Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence, no. 8, vol. 39, AAAI Press, pp. 8736-8744, 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025), Philadelphia, Pennsylvania, United States, 25/02/25. https://doi.org/10.1609/aaai.v39i8.32944

Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration. / Xie, Lianxin; Zheng, Bingbing; Xue, Wen et al.
Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence. ed. / Toby Walsh; Julie Shah; Zico Kolter. AAAI Press, 2025. p. 8736-8744 (Proceedings of the AAAI Conference on Artificial Intelligence; Vol. 39, No. 8).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration

AU - Xie, Lianxin

AU - Zheng, Bingbing

AU - Xue, Wen

AU - Zhang, Yunfei

AU - Jiang, Le

AU - Xu, Ruotao

AU - Wu, Si

AU - Wong, Hau-San

PY - 2025

Y1 - 2025

N2 - Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration. Copyright © 2025, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

AB - Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration. Copyright © 2025, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

UR - https://www.scopus.com/pages/publications/105004301707

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-105004301707&origin=recordpage

U2 - 10.1609/aaai.v39i8.32944

DO - 10.1609/aaai.v39i8.32944

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 157735897

SN - 9781577358978

T3 - Proceedings of the AAAI Conference on Artificial Intelligence

SP - 8736

EP - 8744

BT - Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence

A2 - Walsh, Toby

A2 - Shah, Julie

A2 - Kolter, Zico

PB - AAAI Press

T2 - 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025)

Y2 - 25 February 2025 through 4 March 2025

ER -

Xie L, Zheng B, Xue W, Zhang Y, Jiang L, Xu R et al. Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration. In Walsh T, Shah J, Kolter Z, editors, Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence. AAAI Press. 2025. p. 8736-8744. (Proceedings of the AAAI Conference on Artificial Intelligence; 8). Epub 2025 Apr 11. doi: 10.1609/aaai.v39i8.32944

Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration

Abstract

Publication series

Conference

Funding

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this