Rethinking Semantic Image Compression : Scalable Representation with Cross-modality Transfer

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations


Original languageEnglish
Number of pages6
Journal / PublicationIEEE Transactions on Circuits and Systems for Video Technology
Online published31 Jan 2023
Publication statusOnline published - 31 Jan 2023


This article proposes the scalable cross-modality compression (SCMC) paradigm, in which the image compression problem is further cast into a representation task by hierarchically sketching the image with different modalities. Herein, we adopt the conceptual organization philosophy to model the overwhelmingly complicated visual patterns, based upon the semantic, structure, and signal level representation accounting for different tasks. The SCMC paradigm that incorporates the representation at different granularities supports diverse application scenarios, such as high-level semantic communication and low-level image reconstruction. The decoder, which enables the recovery of the visual information, benefits from the scalable coding based upon the semantic, structure, and signal layers. Qualitative and quantitative results demonstrate that the SCMC can convey accurate semantic and perceptual information of images, especially at low bitrates, and promising rate-distortion performance has been achieved compared to state-of-the-art methods. The code will be available online © 2023 IEEE.

Research Area(s)

  • cross-modality, Data mining, Decoding, Feature extraction, Image coding, Image reconstruction, scalable coding, Semantic image compression, Semantics, Visualization