Spherical Image Inpainting with Frame Transformation and Data-Driven Prior Deep Networks

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

View graph of relations


  • Chaoyan Huang
  • Michael K. Ng
  • Tieyong Zeng

Related Research Unit(s)


Original languageEnglish
Pages (from-to)1179-1196
Journal / PublicationSIAM Journal on Imaging Sciences
Issue number3
Online published24 Jul 2023
Publication statusPublished - 2023



Spherical image processing has been widely applied in many important fields, such as omnidirec-tional vision for autonomous cars, global climate modeling, and medical imaging. It is nontrivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with a deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regular-izer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using a deep learning denoiser and a plug-and-play model. © 2023 Society for Industrial and Applied Mathematics

Research Area(s)

  • spherical image inpainting, deep CNN, plug-and-play, WAVELET ANALYSIS, RECONSTRUCTION

Download Statistics

No data available