Compact Temporal Trajectory Representation for Talking Face Video Compression
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Pages (from-to) | 7009-7023 |
Journal / Publication | IEEE Transactions on Circuits and Systems for Video Technology |
Volume | 33 |
Issue number | 11 |
Online published | 27 Apr 2023 |
Publication status | Published - Nov 2023 |
Link(s)
Abstract
In this paper, we propose to compactly represent the nonlinear dynamics along the temporal trajectories for talking face video compression. By projecting the frames into a high dimensional space, the temporal trajectories of talking face frames, which are complex, non-linear and difficult to extrapolate, are implicitly modelled in an end-to-end inference framework based upon very compact feature representation. As such, the proposed framework is suitable for ultra-low bandwidth video communication and can guarantee the quality of the reconstructed video in such applications. The proposed compression scheme is also robust against large head-pose motions, due to the delicately designed dynamic reference refresh and temporal stabilization mechanisms. Experimental results demonstrate that compared to the state-of-the-art video coding standard Versatile Video Coding (VVC) as well as the latest generative compression schemes, our proposed scheme is superior in terms of both objective and subjective quality at the same bitrate. The project page can be found at https://github.com/Berlin0610/CTTR.
© 2023 IEEE
© 2023 IEEE
Research Area(s)
- Talking face, video compression, visual quality assessment, compact feature representation
Citation Format(s)
Compact Temporal Trajectory Representation for Talking Face Video Compression. / Chen, Bolin; Wang, Zhao; Li, Binzhe et al.
In: IEEE Transactions on Circuits and Systems for Video Technology, Vol. 33, No. 11, 11.2023, p. 7009-7023.
In: IEEE Transactions on Circuits and Systems for Video Technology, Vol. 33, No. 11, 11.2023, p. 7009-7023.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review