Delay-rate-distortion model for real-time video communication

With the increased use of smart phones and mobile tablets, video encoding and streaming over wireless networks become a big concern. Real-time video service requires low end-to-end delay, which mainly consists of video encoding delay, queuing delay, and transmission delay. Previous works either assume video encoding delay to be constant or infinite and neglect it from end-to-end delay. However, encoding time has an effective impact on encoding rate (R) and distortion (D). Hence, the optimal R-D performance based on the assumption of infinite encoding time is neither accurate nor realistic. To achieve the optimal end-to-end performance under low delay constraint, we extend the traditional R-D model to a novel delay-rate-distortion (d-R-D) model, which quantifies the relationship among source coding time, rate, and distortion for IPPPP coding mode in H.264/Advanced Video Coding (AVC). On the other hand, a delay insensitive video communication application can afford hierarchical bidirectional prediction mode that introduces extra encoding delay than IPPPP mode in H.264/AVC, as it achieves higher encoding efficiency when delay is not a top priority. However, coding efficiency does not always increase with delay, and delay is not unlimited as well. How to maximize coding efficiency in hierarchical B mode with limited encoding time constraint is not a well-addressed problem. With the proposed d-R-D model, we can mathematically formulate this problem and provide a practical solution for the first time.