A Non-Local CNN for Video Denoising

Axel Davy; Thibaud Ehret; Jean-Michel Morel; Pablo Arias; Gabriele Facciolo

doi:10.1109/ICIP.2019.8803314

A Non-Local CNN for Video Denoising

Axel Davy
, Thibaud Ehret
, Jean-Michel Morel
, Pablo Arias
, Gabriele Facciolo

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

68 Citations (Scopus)

Abstract

Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising. © 2019 IEEE.

Original language	English
Title of host publication	2019 IEEE International Conference on Image Processing - Proceedings
Publisher	IEEE
Pages	2409-2413
ISBN (Electronic)	9781538662496
ISBN (Print)	9781538662502
DOIs	https://doi.org/10.1109/ICIP.2019.8803314
Publication status	Published - 2019
Externally published	Yes
Event	26th IEEE International Conference on Image Processing (ICIP 2019) - Taipei International Convention Center (TICC), Taipei, Taiwan, China Duration: 22 Sept 2019 → 25 Sept 2019

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2019-September
ISSN (Print)	1522-4880
ISSN (Electronic)	2381-8549

Conference

Conference	26th IEEE International Conference on Image Processing (ICIP 2019)
Abbreviated title	IEEE ICIP 2019
Place	Taiwan, China
City	Taipei
Period	22/09/19 → 25/09/19

Funding

Work supported by IDEX Paris-Saclay IDI 2016, ANR-11-IDEX-0003- 02, ONR grant N00014-17-1-2552, CNES MISS project, DGA Astrid ANR17-ASTR-0013-01, DGA ANR-16-DEFA-0004-01. The TITAN V used for this research was donated by the NVIDIA Corporation.

Research Keywords

Convolutional Neural Networks
Denoising
Non-locality
Video Processing

Access Output

10.1109/ICIP.2019.8803314

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{811983c6796d4a22a546a7a9e3c1f905,

title = "A Non-Local CNN for Video Denoising",

abstract = "Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising. {\textcopyright} 2019 IEEE.",

keywords = "Convolutional Neural Networks, Denoising, Non-locality, Video Processing",

author = "Axel Davy and Thibaud Ehret and Jean-Michel Morel and Pablo Arias and Gabriele Facciolo",

year = "2019",

doi = "10.1109/ICIP.2019.8803314",

language = "English",

isbn = "9781538662502",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE",

pages = "2409--2413",

booktitle = "2019 IEEE International Conference on Image Processing - Proceedings",

address = "United States",

note = "26th IEEE International Conference on Image Processing (ICIP 2019), IEEE ICIP 2019 ; Conference date: 22-09-2019 Through 25-09-2019",

}

Davy, A, Ehret, T, Morel, J-M, Arias, P & Facciolo, G 2019, A Non-Local CNN for Video Denoising. in 2019 IEEE International Conference on Image Processing - Proceedings., 8803314, Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, IEEE, pp. 2409-2413, 26th IEEE International Conference on Image Processing (ICIP 2019), Taipei, Taiwan, China, 22/09/19. https://doi.org/10.1109/ICIP.2019.8803314

A Non-Local CNN for Video Denoising. / Davy, Axel; Ehret, Thibaud; Morel, Jean-Michel et al.
2019 IEEE International Conference on Image Processing - Proceedings. IEEE, 2019. p. 2409-2413 8803314 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - A Non-Local CNN for Video Denoising

AU - Davy, Axel

AU - Ehret, Thibaud

AU - Morel, Jean-Michel

AU - Arias, Pablo

AU - Facciolo, Gabriele

PY - 2019

Y1 - 2019

N2 - Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising. © 2019 IEEE.

AB - Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising. © 2019 IEEE.

KW - Convolutional Neural Networks

KW - Denoising

KW - Non-locality

KW - Video Processing

UR - https://www.scopus.com/pages/publications/85076807040

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85076807040&origin=recordpage

U2 - 10.1109/ICIP.2019.8803314

DO - 10.1109/ICIP.2019.8803314

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781538662502

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 2409

EP - 2413

BT - 2019 IEEE International Conference on Image Processing - Proceedings

PB - IEEE

T2 - 26th IEEE International Conference on Image Processing (ICIP 2019)

Y2 - 22 September 2019 through 25 September 2019

ER -

A Non-Local CNN for Video Denoising

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this