Learning-based Complexity Control for High Efficiency Video Coding and Beyond

Description

In June 6th 2016, Cisco released the White paper[1], VNI Forecast and Methodology2015-2020, reported that 82 percent of Internet traffic will come from video applicationssuch as video surveillance, content delivery network, so on by 2020. It also reported thatInternet video surveillance traffic nearly doubled, Virtual reality traffic quadrupled, TVgrew 50 percent and similar increases for other applications in 2015. The annual globaltraffic will first time exceed the zettabyte(ZB;1000 exabytes[EB]) threshold in 2016, andwill reach 2.3 ZB by 2020. It implies that 1.886ZB belongs to video data.Thus, in order to relieve the burden on video storage, streaming and other video services,researchers from the video community have developed a series of video coding standards.Among them, the most up-to-date is the High Efficiency Video Coding(HEVC) or H.265standard, which has successfully halved the coding bits of its predecessor, H.264/AVC,without significant increase in perceived distortion.Rate-Distortion (RD) has been commonly adopted as a metric to measure theeffectiveness of HEVC based video coder. However, Rate-Distortion improvement fromthe new tools proposed by HEVC such as more coding unit partitions, and larger motionestimation ranges have inevitably resulted in a dramatic growth of coding complexity,which further limits the industrial deployment of HEVC, especially in power-limited andreal-time applications. Many fast video coding algorithms have been developed toaddress this problem. However, these algorithms can usually only resolve a complexityreduction for a specific video configuration set, but not different application scenariosand systems, such as video coding on power-limited mobile devices, real-time videobroadcasting, and so on.In this research, we will consider coding Complexity(C) as an important criterion in theholistic performance of HEVC and develop complexity control techniques that flexiblyadapt to systems with different computation resources. We propose to develop alearning-based complexity control framework with advanced machine learning methodson sample selection and regression prediction. This learning-based framework can solveoptimization problems in video coding by discovering the relationships between featuresfrom video contents and different prediction performances. As a result, the proposedcontrollable coding scheme can adapt to diverse video application scenarios.Furthermore, it is expected that our research will promote the use of complexity controlas an essential ingredient to video optimization and flexible video coding for HEVC andits successors. The developed technology is expected to drive the future of digital mediaindustry.[1] http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visualnetworking-index-vni/complete-white-paper-c11-481360.html?