Design Considerations for Efficient Video Encoders Based on Spatial-Temporal-View for Multiview Video Systems

Multi-view (or multi-camera) video has attracted much attention recently. Utilized in a wide range of multimedia applications, such as immersive games, movies, presentations, video conferencing, 3-D TV and medical imaging, it consists of video sequences (of the same scenario) captured by multiple cameras, but from different angles and locations, resulting in the need to store and/or transmit tremendous amounts of data. This subsequently induces a large amount of inter-view statistical dependencies and redundancies in multi-view video. Efficient encoding/compression technique is vital for the success of multi-view video. Multi-view Video Coding (MVC) has been recently developed based on the extension H.264/AVC. The goal of MVC is, by efficiently exploring not only temporal but also inter-view redundancies, to provide higher coding performance than the independent mono-view coding. However, the coding performance improvement is at the cost of dramatically increasing encoding complexity due to the rate-distortion optimized temporal/inter-view prediction structure and variable-block-size mode decision. It is imperative to design optimization approaches which remove the computational obstacles for MVC. In this project, several innovative techniques are proposed to overcome the computational problem, including early termination methods for motion/disparity estimation and mode decision, fast motion/disparity estimation algorithms and fast mode decision algorithms. This will enable MVC more applicable to niche markets such as three-dimensional TV, free viewpoint TV, holographic imaging, etc., resulting in valuable outputs, including research publications, software packages and a multi-camera video capturing system.