An efficient mode decision algorithm for H.264/AVC encoding optimization

The H.264 video coding standard significantly outperforms previous standards in terms of coding efficiency. However, this comes as a cost of extremely high computational complexity due to mode decision where variable block size Motion Estimation (ME) is employed. In this paper, we propose an efficient algorithm to jointly optimize mode decision and ME. A theoretical analysis is performed to study the sufficient condition to detect all-zero blocks in H.264, and thus adaptive thresholds are derived to early terminate mode decision and ME. Besides the aforementioned early termination technique, the proposed algorithm also introduces temporal-spatial checking, thresholds based prediction and monotonic error surface based prediction methods to skip checking unnecessary modes. Experimental results demonstrate that the proposed algorithm can significantly reduce the computational complexity of H.264 encoding while maintaining almost the same Rate Distortion (RD) performance as the original encoder. © 2007 IEEE.