Rate-distortion based vector quantization technique and H.264/AVC video coding standard for video compression

基於率失真的向量量化技術和 H.264/AVC 視頻編碼標準在視頻壓縮中的應用

Student thesis: Master's Thesis

View graph of relations


  • Kai GUO

Related Research Unit(s)


Awarding Institution
Award date2 Oct 2007


With the widespread adoption of video technologies, video compression has been a hot topic that can effectively handle the redundancies in video data. There are two primary video compression techniques: vector quantization (VQ) and hybrid coding scheme. VQ is a powerful data compression technique since it is theoretically attractive; however, there still exists a large gap between the theoretical performance and the actual achieved performance. In this thesis, a new adaptive vector quantization (AVQ) technique using the partial codevector updating (PCU) scheme is proposed. The PCU scheme is based on rate-distortion optimization (RDO) and achieves rate-distortion performance superior to that of the conventional AVQ algorithms using the full codevector updating (FCU) scheme. The PCU-AVQ algorithm only updates the codevector’s components with quantization errors higher than an optimal threshold instead of replacing the whole codevector. Besides, the mathematical relation between the Lagrangian multiplier and the approximate optimal threshold is devised to reduce the rate-distortion cost computation. In addition, a fast PCU-AVQ algorithm is also proposed to reduce the encoding complexity. In order to adapt to the changing source statistics during the coding process, a multiple codebook system is proposed to represent the diverse feature of source data. In this thesis a specific multiple codebook system is employed for video coding, which can efficiently reduce the spatial and temporal redundancies. Hybrid coding scheme is the basis of all the video coding standards, including the newest H.264/AVC, which greatly outperforms the previous MPEG-1/2/4 and H.261/263 standards in terms of both picture quality and compression efficiency. In order to achieve the highest coding efficiency, H.264 employs RDO technique to determine the best mode which leads to the optimal tradeoff between the coding quality and consumed bit rate. However, this mode decision process also introduces extremely high complexity in the encoding process, including the computation of the sum of squared differences (SSD) between the original and reconstructed image blocks. In this thesis, a fast SSD (FSSD) algorithm is proposed to reduce the complexity of the rate-distortion cost function implementation. The proposed FSSD algorithm is based on the theoretical equivalence of the SSD in spatial and transform domains and determines the distortion in integer cosine transform domain using an iterative table-lookup quantization process. This approach could avoid the inverse quantization/transform and pixel reconstructions processes with nearly no rate-distortion performance degradation. In addition, the FSSD algorithm can also be used with efficient bit-rate estimation algorithms to further reduce the cost function complexity. In order to further reduce the coding complexity spent on mode decision, a new rate-based inter mode decision algorithm is proposed. The main idea is to classify each macroblock into simple motion or complex motion content based on the bit rate of Inter16x16 mode’s residue block in order to avoid checking some unlikely modes. Simulation results indicate that the proposed mode selection algorithm can efficiently reduce the computational complexity with a little degradation of the rate-distortion performance.

    Research areas

  • Digital techniques, Signal processing, Video compression, Standards