Learning the Scale in Reference Picture Resampling for Versatile Video Coding

Riyu Lu, Yingwen Zhang, Hengyu Man, Meng Wang, Shiqi Wang*, Xiaopeng Fan*

*Corresponding author for this work

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

Abstract

Compressing high-resolution videos under low bitrate constraints is a challenging task. Resampling-based compression, which reduces the resolution before encoding and restores it after decoding, has great potential to improve the rate-distortion performance in such scenarios. In this paper, we propose a learning-based frame-level coding scale control scheme that enhances the coding performance by adjusting the coding scale for each frame. The scheme cooperates with the Reference Picture Resampling of the latest video coding standard Versatile Video Coding (VVC), which allows coding scale variations on each frame. More specifically, a dataset with 5200 videos is created by a greedy rate-distortion optimization algorithm employed to select the optimal coding scale for each frame. A neural network-based decision model is further incorporated into VVC, learning to predict the coding scale for each frame in one pass. The scheme is implemented into the Fraunhofer Versatile Video Encoder (VVenC), a fast and efficient VVC encoder, and evaluated on 4K contents. Experimental results show that the proposed scheme outperforms GOP-based coding scale adaptation methods, achieving average bitrate savings of 3.06% and 4.14% in terms of PSNR and MS-SSIM. © 2025 IEEE.
Original languageEnglish
JournalIEEE Transactions on Multimedia
DOIs
Publication statusOnline published - 17 Feb 2025

Funding

This work was supported in part by the National Key RD Program of China (2021YFF0900500), the National Natural Science Foundation of China (NSFC) under grants U22B2035 and 62272128, in part by ITF Project GHP/044/21SZ and in part by RGC General Research Fund 11203220/11200323.

Research Keywords

  • rate-distortion optimization
  • resampling-based compression
  • versatile video coding
  • Video coding

Fingerprint

Dive into the research topics of 'Learning the Scale in Reference Picture Resampling for Versatile Video Coding'. Together they form a unique fingerprint.

Cite this