Faster and Stronger Lossless Compression with Optimized Autoregressive Framework

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publication2023 60th ACM/IEEE Design Automation Conference (DAC)
PublisherInstitute of Electrical and Electronics Engineers, Inc.
Number of pages6
ISBN (electronic)979-8-3503-2348-1
ISBN (print)979-8-3503-2349-8
Publication statusPublished - 2023

Conference

Title60th Design Automation Conference (DAC 2023)
PlaceUnited States
CitySan Francisco
Period9 - 13 July 2023

Abstract

Neural AutoRegressive (AR) framework has been applied in general-purpose lossless compression recently to improve compression performance. However, this paper found that directly applying the original AR framework causes the duplicated processing problem and the in-batch distribution variation problem, which leads to deteriorated compression performance. The key to address the duplicated processing problem is to disentangle the processing of the history symbol set at the input side. Two new types of neural blocks are first proposed. An individual-block performs separate feature extraction on each history symbol while a mix-block models the correlation between extracted features and estimates the probability. A progressive AR-based compression framework (PAC) is then proposed, which only requires one history symbol from the host at a time rather than the whole history symbol set. In addition, we introduced a trainable matrix multiplication to model the ordered importance, replacing previous hardware-unfriendly Gumble-Softmax sampling. The in-batch distribution variation problem is caused by AR-based compression’s structured batch construction. Based on this observation, a batch-location-aware individual block is proposed to capture the heterogeneous in-batch distributions precisely, improving the performance without efficiency losses. Experimental results show the proposed framework can achieve an average of 130% speed improvement with an average of 3% compression ratio gain across data domains compared to the state-of-the-art. © 2023 IEEE.

Research Area(s)

  • auto-regressive, general-purpose, lossless data compression, hardware friendly, neural networks, computational efficient

Bibliographic Note

Information for this record is supplemented by the author(s) concerned.

Citation Format(s)

Faster and Stronger Lossless Compression with Optimized Autoregressive Framework. / Mao, Yu; Li, Jingzong; Cui, Yufei et al.
2023 60th ACM/IEEE Design Automation Conference (DAC). Institute of Electrical and Electronics Engineers, Inc., 2023.

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review