An efficient mode decision algorithm for H.264/AVC encoding optimization

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

88 Scopus Citations
View graph of relations


Related Research Unit(s)


Original languageEnglish
Pages (from-to)882-888
Journal / PublicationIEEE Transactions on Multimedia
Issue number4
Publication statusPublished - Jun 2007


The H.264 video coding standard significantly outperforms previous standards in terms of coding efficiency. However, this comes as a cost of extremely high computational complexity due to mode decision where variable block size Motion Estimation (ME) is employed. In this paper, we propose an efficient algorithm to jointly optimize mode decision and ME. A theoretical analysis is performed to study the sufficient condition to detect all-zero blocks in H.264, and thus adaptive thresholds are derived to early terminate mode decision and ME. Besides the aforementioned early termination technique, the proposed algorithm also introduces temporal-spatial checking, thresholds based prediction and monotonic error surface based prediction methods to skip checking unnecessary modes. Experimental results demonstrate that the proposed algorithm can significantly reduce the computational complexity of H.264 encoding while maintaining almost the same Rate Distortion (RD) performance as the original encoder. © 2007 IEEE.

Research Area(s)

  • Early termination, H.264, Integer DCT, Mode decision, Motion estimation