From Source-available to Source-free Unsupervised Prototypical Domain Adaptation for Lesion Segmentation

Project: Research

View graph of relations


Wireless capsule endoscopy (WCE) is an effective technique to detect colorectal lesions, which are precursors of colorectal cancer (CRC). Automatic lesion segmentation from WCE images is of great importance to provide valuable information for both CRC diagnosis and further surgery. Deep learning based approaches have achieved significant progress but at the expense of laborious large-scale data annotation. One possible solution to address this issue is unsupervised domain adaptation, which transfers knowledge from labeled datasets in a source domain to an unlabeled target domain. In this proposal, we aim to tackle the important but technically challenging problem of efficiently utilizing the public available colonoscopy images (source domain) to guide lesion segmentation with WCE images (target domain). There are several challenges in applying existing domain adaptation models for lesion segmentation. (1) The majority of these methods directly minimize feature distances in two domains, to enable the model trained with source images to obtain superior performance in the target domain. But WCE images inevitably include noise with inaccurate features due to complex imaging situations, which may deviate the segmentation performance. (2) These models lack comprehensive feature representations and explicit structure relationships between the source domain and the target domain. (3) Current domain adaptation approaches assume that samples from source and target domains are freely accessible during the training phase. However, this assumption is usually impractical in medical-related applications due to data privacy issues. To deal with these challenges, we propose comprehensive unsupervised domain adaptation models from source-available to source-free for automatic lesion segmentation. Firstly, we propose a hierarchical prototype based domain adaptation model with a self-training strategy to refine features, thus mitigating noise issues. Secondly, learnable graph structures are introduced to the hierarchical model exploiting intrinsic structural relationships among multi-scale features and prototypes of different domains. Moreover, we propose a source-free prototypical domain adaptation model capable of exploiting unlabeled WCE images and the pre-trained model with colonoscopy images, to deal with data privacy issues. Finally, we will evaluate and validate the proposed models with collected WCE images and videos, and implement a graphical user interface to display segmentation results. Our preliminary experiments have already demonstrated both the feasibility and the effectiveness of the proposed models. This project represents the first attempt to develop domain adaptation models for lesion segmentation with WCE images. It will produce high impacts on academic research of medical image analysis, and provide practical solutions to precision medicine in society and industry. 


Project number9043152
Grant typeGRF
Effective start/end date1/01/22 → …