Intra predictions and mode decisions for intra coding in H.264/AVC

Abstract

H.264/AVC is the latest international video coding standard issued by International Organization for Standardization (ISO) and International Telecommunication Union (ITU). Compared with the previous standards, it can achieve more than double compression efficiency. One of the main contributions of H.264/AVC is that it introduces the intra prediction in spatial domain, which increases the I-frame compression ratio significantly. In spite of the good performance, H.264/AVC still has some weaknesses that can be further improved. In this thesis, new intra prediction and mode decision techniques are proposed for intra coding in H.264/AVC. The Lagrangian rate distortion optimization (RDO) in mode decision is an important technique of H.264/AVC. In this technique, RDO cost function is used to select the best mode based on both coding quality and bitrate, which greatly increases the overall coding performance. However, this technique involves high computational load. The first work of this thesis is to propose an enhanced sum of absolute integer transform difference (ESAITD) cost function, which adopts a simple bitrate predictor called "sum of non-zero coefficient frequencies". This rate predictor uses simple but efficient parameter to approximate the actual encoded bitrate. Experimental results show that using ESAITD achieves better rate-distortion (RD) performance than using sum of absolute Hadamard transform difference (SATD) cost function. It also outperforms using sum of absolute integer transform difference (SAITD) cost function in medium and high bitrate cases. The second work of this thesis is to propose a novel weighted cross prediction (WCP) to simplify the computational complexity of distance-based weighted prediction (DWP) method. In WCP mode, the upper right part of one 4x4 block mainly employs vertical prediction while the lower left part mainly uses horizontal prediction. The fixed weighted coefficients in our approach result in lower computational complexity and remain a comparable performance with DWP. The third work of this thesis focuses on the reduction of intra prediction mode bits. This work is composed of three parts. In the first part, an adaptive intra mode bit skip (AIMBS) with DWP (AIMBS-DWP) method is introduced. The AIMBS is a technique to improve the efficiency of Intra_4x4 coding in H.264/AVC by considering the smoothness of boundary pixels. However, the Direct-Current (DC) mode in AIMBS becomes ineffective in predicting the blocks without directional preferences. To tackle this problem and further improve the coding efficiency, DWP mode is proposed to replace DC mode in the Multiple-Prediction of AIMBS. It enhances the robustness of AIMBS in larger range of QP and achieves higher rate distortion (RD) performance. The second part aims at reducing the prediction mode bits by increasing the hit rate of most probable mode (MPM). For that purpose, a recent method, which is called L-shaped template MPM estimation (LMPME) method, is studied. The LMPME method can apparently increase the hit rate of MPM by considering an L-shaped template around the current block, but it is computationally complex. Based on some analysis and experimental results, a simplified version of LMPME method is proposed. It is shown that the simplified LMPME (SLMPME) method has a comparable performance with the original version, while the computational complexity is reduced. Finally, the combinations of the AIMBS-DWP method with LMPME/SLMPME methods are presented. Experimental results show that the combination of these three methods improves the performance largely.

Date of Award	4 Oct 2010
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Lai Man PO (Supervisor)