Rendering virtual objects with real world surface into real scene

Abstract

In the recent decade, Image Based techniques and their related applications have become a hot topic. There are many commercial products such as movies and Augmented Reality applications, which use these techniques. Image-Based techniques can produce a more photorealistic image, and can smoothly merge real and virtual objects together within a scene. Another reason causing the growth in this technique is that hardware limitations have been virtually eliminated in recent years. Nowadays, Image-Based techniques can be implemented in a powerful single or distributed machine. This research project is focused on seamless integration and improvement of Image Based techniques. The input element for the proposed system is a set of photos of the real scene and of its additional objects, which are taken from a combination of different viewpoints and lightings angles, and enable the method to capture the intrinsic parameters of the scene such as: object structure, object surface properties, lighting direction, etc. In the first stage, the geometric information of image objects is extracted by 3d techniques, so that it can reconstruct a virtual object from the real scene with captured photo in different viewpoints. In the second stage the method presents a new and novel surface reconstruction technique, which applies 2d Delaunay triangulation in 3d space, and reconstructs a smooth surface over a dense set of points generated from the 3d reconstruction. These first two techniques are presented as an integrated image based modeling approach. In the third stage, a reflectance recovery technique employs high dynamic range radiance maps together with the environment radiance to recreate the surface properties of scene objects from the image set. Within this technique, an enhanced iterative technique is devised to recover the diffuse and specular parameters of each object in the scene, together with the most significant parameters and color differences between the images being resolved. In the final stage, this research offers an approach to rendering the new surface into the real scene, using a public real world surface texture database within which a “matching and pasting” approach had been developed to recreate the effect of natural surfaces from the texture database. As a result, the newly developed virtual object is rendered into the real scene, and the combined scene re-rendered for different observation point, exposure, virtual lighting or even surface parameters on the new scene. In summary, this research combines the existing advantages of modeling and rendering techniques to achieve an integrated system for Image-Based method in a way which is efficient and easy to use, and can help people flexibly and efficiently generate photo-realistic images.

Date of Award	15 Jul 2004
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Yu MONG (Supervisor)