Complexity reduced mode selection of H.264/AVC intra coding

One of the new features in the H.264/AVC encoder is the use of Lagrangian Rate-Distortion Optimization (RDO) method during mode decision at the macroblock level. The RDO technique has been employed in H.264/AVC for intra prediction mode selection to achieve better coding efficiency. But the computation complexity of mode decision algorithm is extremely high. To reduce the complexity of mode decision, we propose an efficient and fast 4×4 intra prediction mode selection scheme. The proposed method reduces the candidate of the prediction modes based on the Sum of Absolute Transformed Difference (SATD) between the original block and the intra predicted block. Rank of each mode is obtained based on the SATD value. The candidate modes are further reduced by using the combination of rank and most probable mode. Simulation results demonstrate that the proposed mode decision method reduces about 95 % of mode decision time of intra coding with ignorable degradation of coding performance. © 2008 IEEE.