Fast Rate-Distortion Cost Functions Computation for High-Quality H.264/AVC Video Encoding

Project: Research

View graph of relations


The newest H.264/AVC video coding standard achieves much higher compression performance than the previous standards, such as MPEG-1/2/4 and H.261/H.263. This new standard not only pursues higher quality, but also opens new opportunities for various bandwidth demanding video applications, especially for high-definition and high-quality digital video applications. To achieve the highest coding efficiency, H.264 employs a rate-distortion optimization technique to get the optimal coding results, in which the encoder selects the best coding mode by computing all the possible modes. However, this mode decision process has introduced extremely high complexity into the encoding process, especially the cost function’s sum of squared difference (SSD) computation. Hence, the cost function computation makes H.264 impossible to be realized in real-time applications without high computing hardware. To tackle this problem, many fast H.264 mode decision algorithms have been developed in recent years. All of these developed fast algorithms, however, have the major drawback of quite significant performance degradation in terms of both coding bit-rate and reconstructed video quality, which may not be acceptable for the coming high-definition digital video era. This project will have a significant impact on software and hardware implementation of the high quality H.264 video encoder. A new concept on computing cost function’s SSD in integer cosine transform domain will be explored. This approach will not introduce any rate-distortion performance degradation in the mode decision process as it is based on the theoretical equivalent of the SSDs in spatial and transform domains.


Project number9041251
Grant typeGRF
Effective start/end date1/10/078/02/10