Video completion and synthesis

This paper presents a new exemplar-based framework for video completion, allowing aesthetically pleasing completion of large space-time holes. We regard video completion as a discrete global optimization on a 3D graph embedded in the space-time video volume. We introduce a new objective function which enforces global spatio-temporal consistency among patches that fill the hole and surrounding it, in terms of both color similarity and motion similarity. The optimization is solved by a novel algorithm, called weighted priority belief propagation (BP), which alleviates the problems of slow convergence and intolerable storage size when using the standard BP. This objective function can also handle video texture synthesis by extending an input video texture to a larger texture region. Experiments on a wide variety of video examples with complex dynamic scenes demonstrate the advantages of our method over existing techniques: salient structures and motion information are much better restored. Copyright ©2008 John Wiley & Sons, Ltd.