Variational Foundation for Geometric Image Representation

Image representation reflects how visual information is processed through perception, and the functional perspective of images leads to variational approaches in computational theory of vision. Mumford and Shah proposed a representation characterized by smooth regions separated by finitely many continuously differentiable boundaries. Although the Mumford and Shah (MS) model has been widely applied, it does not predict image features of all dimensions, including 0-dimensional curvature extrema and X-junctions. An alternative approach to enrich the model involves the explicit construction of the desired geometric elements, an approach largely relying on heuristics. This significant gap cannot be easily bridged by traditional Finite Element Method (FEM) for the MS functional nor by recent developments in network-based techniques. In this project, we focus on the variational foundation of geometric image representation. Our work advances the classical MS functional towards composability and controllability. The proposed new models will define and use piecewise smooth multidimensional primitives to characterize the interaction between the different image features. Our first goal is to establish novel variational frameworks for image representation containing critical visual cues across all dimensions. To allow for explicit modeling for these elements, we will develop our models in a subspace of SBV, the space of special functions with bounded variation initially introduced by the de Giorgi school to solve the Mumford-Shah conjecture. Notably, we aim for models with explicit regularization on representational complexity. This framework will improve our previous formalization for piecewise constant vector graphics and extend to piecewise smooth images. Our second goal is to propose an effective scheme to generate quadrilateral meshes which are compatible with image singularities. It bypasses the obstruction from two-dimensional patches to one-dimensional singular set of the traditional methods. Finally, we plan to develop publicly available programs accompanied with accessible online demos.