Controllable Image Processing via Adaptive FilterBank Pyramid

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

View graph of relations

Author(s)

  • Dongdong Chen
  • Qingnan Fan
  • Angelica Aviles-Rivero
  • Lu Yuan
  • Nenghai Yu
  • Gang Hua

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number9146380
Pages (from-to)8043-8054
Number of pages12
Journal / PublicationIEEE Transactions on Image Processing
Volume29
Online published22 Jul 2020
Publication statusPublished - 2020

Abstract

Traditional image processing operators often provide some control parameters to tweak the final results. Recently, different convolutional neural networks have been used to approximate or improve these operators. However, in those methods, one single model can only handle one operator of a specific parameter value and does not support parameter tuning. In this paper, we propose a new plugin module, "Adaptive Filterbank Pyramid", which can be inserted into a backbone network to support multiple operators and continuous parameter tuning. Our module explicitly represents one operator with one filterbank pyramid. To generate the results of a specific operator, the corresponding filterbank pyramid is convolved with the intermediate feature pyramid produced by the backbone network. The weights of the filterbank pyramid are directly regressed by another sub-network, which is jointly trained with the backbone network and adapted to the input parameter, thus enabling continuous parameter tuning. We applied the proposed module for a large variety of image processing tasks, including image smoothing, image denoising, image deblocking, image enhancement and neural style transfer. Experiments show that our method is generalized to different types of image processing tasks and different backbone network structures. Compared to the single-operator-single-parameter baseline, our method can produce comparable results but is significantly more efficient in both training and testing.

Research Area(s)

  • adaptive filterbank, Controllable image processing, image restoration

Citation Format(s)

Controllable Image Processing via Adaptive FilterBank Pyramid. / Chen, Dongdong; Fan, Qingnan; Liao, Jing; Aviles-Rivero, Angelica; Yuan, Lu; Yu, Nenghai; Hua, Gang.

In: IEEE Transactions on Image Processing, Vol. 29, 9146380, 2020, p. 8043-8054.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal