Automatic Polyp Recognition in Colonoscopy Images Using Deep Learning and Two-Stage Pyramidal Feature Prediction

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

14 Scopus Citations
View graph of relations

Author(s)

  • Xiao Jia
  • Xiaochun Mai
  • Yi Cui
  • Xiaohan Xing
  • Hyunseok Seo
  • Lei Xing
  • Max Q.-H. Meng

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)1570-1584
Number of pages15
Journal / PublicationIEEE Transactions on Automation Science and Engineering
Volume17
Issue number3
Online published30 Jan 2020
Publication statusPublished - Jul 2020

Abstract

Polyp recognition in colonoscopy images is crucial for early colorectal cancer detection and treatment. However, the current manual review requires undivided concentration of the gastroenterologist and is prone to diagnostic errors. In this article, we present an effective, two-stage approach called PLPNet, where the abbreviation “PLP” stands for the word “polyp,” for automated pixel-accurate polyp recognition in colonoscopy images using very deep convolutional neural networks (CNNs). Compared to hand-engineered approaches and previous neural network architectures, our PLPNet model improves recognition accuracy by adding a polyp proposal stage that predicts the location box with polyp presence. Several schemes are proposed to ensure the model’s performance. First of all, we construct a polyp proposal stage as an extension of the faster R-CNN, which performs as a region-level polyp detector to recognize the lesion area as a whole and constitutes stage I of PLPNet. Second, stage II of PLPNet is built in a fully convolutional fashion for pixelwise segmentation. We define a feature sharing strategy to transfer the learned semantics of polyp proposals to the segmentation task of stage II, which is proven to be highly capable of guiding the learning process and improve recognition accuracy. Additionally, we design skip schemes to enrich the feature scales and thus allow the model to generate detailed segmentation predictions. For accurate recognition, the advanced residual nets and feature pyramids are adopted to seek deeper and richer semantics at all network levels. Finally, we construct a two-stage framework for training and run our model convolutionally via a single-stream network at inference time to efficiently output the polyp mask. Experimental results on public data sets of GIANA Challenge demonstrate the accuracy gains of our approach, which surpasses previous state-of-the-art methods on the polyp segmentation task (74.7 Jaccard Index) and establishes new top results in the polyp localization challenge (81.7 recall).

Research Area(s)

  • Deep residual network (ResNet), feature pyramids, PLPNet, polyp recognition, two-stage framework

Citation Format(s)

Automatic Polyp Recognition in Colonoscopy Images Using Deep Learning and Two-Stage Pyramidal Feature Prediction. / Jia, Xiao; Mai, Xiaochun; Cui, Yi; Yuan, Yixuan; Xing, Xiaohan; Seo, Hyunseok; Xing, Lei; Meng, Max Q.-H.

In: IEEE Transactions on Automation Science and Engineering, Vol. 17, No. 3, 07.2020, p. 1570-1584.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review