DSI-Net : Deep Synergistic Interaction Network for Joint Classification and Segmentation with Endoscope Images
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Journal / Publication | IEEE Transactions on Medical Imaging |
Publication status | Online published - 25 May 2021 |
Link(s)
Abstract
Automatic classification and segmentation of wireless capsule endoscope (WCE) images are two clinically significant and relevant tasks in a computer-aided diagnosis system for gastrointestinal diseases. Most of existing approaches, however, considered these two tasks individually and ignored their complementary information, leading to limited performance. To overcome this bottleneck, we propose a deep synergistic interaction network (DSI-Net) for joint classification and segmentation with WCE images, which mainly consists of the classification branch (C-Branch), the coarse segmentation (CS-Branch) and the fine segmentation branches (FS-Branch). In order to facilitate the classification task with the segmentation knowledge, a lesion location mining (LLM) module is devised in C-Branch to accurately highlight lesion regions through mining neglected lesion areas and erasing misclassified background areas. To assist the segmentation task with the classification prior, we propose a category-guided feature generation (CFG) module in FS-Branch to improve pixel representation by leveraging the category prototypes of C-Branch to obtain the category-aware features. In such way, these modules enable the deep synergistic interaction between these two tasks. In addition, we introduce a task interaction loss to enhance the mutual supervision between the classification and segmentation tasks and guarantee the consistency of their predictions. Relying on the proposed deep synergistic interaction mechanism, DSI-Net achieves superior classification and segmentation performance on public dataset in comparison with state-of-the-art methods. The source code is available at https://github.com/CityU-AIM-Group/DSI-Net.
Research Area(s)
- Wireless capsule endoscopy, Joint classification and segmentation, Category-guided feature, Lesion location mining
Citation Format(s)
DSI-Net: Deep Synergistic Interaction Network for Joint Classification and Segmentation with Endoscope Images. / Zhu, Meilu; Chen, Zhen; Yuan, Yixuan.
In: IEEE Transactions on Medical Imaging, 25.05.2021.
In: IEEE Transactions on Medical Imaging, 25.05.2021.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review