Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)

33 Scopus Citations
View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Subtitle of host publicationCVPR 2018
PublisherIEEE
Pages6286-6295
Number of pages10
ISBN (Electronic)9781538664209
ISBN (Print)9781538664216
Publication statusPublished - Jun 2018
Externally publishedYes

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

Title31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)
LocationCalvin L. Rampton Salt Palace Convention Center
PlaceUnited States
CitySalt Lake City
Period18 - 22 June 2018

Abstract

Rain removal is important for improving the robustness of outdoor vision based systems. Current rain removal methods show limitations either for complex dynamic scenes shot from fast moving cameras, or under torrential rain fall with opaque occlusions. We propose a novel derain algorithm, which applies superpixel (SP) segmentation to decompose the scene into depth consistent units. Alignment of scene contents are done at the SP level, which proves to be robust towards rain occlusion and fast camera motion. Two alignment output tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match tensor, provide informative clues for rain streak location and occluded background contents to generate an intermediate derain output. These tensors will be subsequently prepared as input features for a convolutional neural network to restore high frequency details to the intermediate output for compensation of misalignment blur. Extensive evaluations show that up to 5dB reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual inspection shows that much cleaner rain removal is achieved especially for highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.

Bibliographic Note

Research Unit(s) information for this record is provided by the author(s) concerned.

Citation Format(s)

Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework. / Chen, Jie; Tan, Cheen-Hau; Hou, Junhui; Chau, Lap-Pui; Li, He.

Proceedings: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition: CVPR 2018. IEEE, 2018. p. 6286-6295 8578756 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)