Domain Knowledge Driven Deep Models for Automatic Gastrointestinal Disease Diagnosis

Project: Research

View graph of relations


Gastrointestinal (GI) diseases pose a great threat to public health in the world. They may last for several years, then deteriorate to cancers, but early detection and treatment of these diseases can significantly reduce the possible occurrence of cancers. Wireless capsule endoscopy (WCE) has emerged as an important diagnostic tool for GI disease detection and therapeutic monitoring, thanks to its non-invasive, user-friendly and non-painful properties. A challenge in harnessing the WCE is that it requires clinicians to analyze a huge number of images (about 50,000 images per patient). Recently, deep learning based approaches have achieved significant progresses in automatic GI disease diagnosis. However, several challenges remain to be resolved for clinical applications. (1) The majority of existing models utilize whole images as inputs, which may introduce noise and redundant background information for feature learning. (2) Current approaches treat WCE images the same as conventional natural images by naively ignoring critical clinical information, often leading to biased conclusions. For instance, the bleeding areas show different colors while polyps have specific shape information. (3) These deep models usually address diagnosis in a supervised manner, requiring a large amount of training data with labels. Unfortunately, labeled medical images are usually difficult or time-consuming to acquire due to privacy issues. To deal with these challenges, we propose domain knowledge driven deep models for automatic GI disease diagnosis in this proposal. Firstly, we propose a hierarchical attention model to help the network focus on abnormal regions and thus improve its ability for discriminative representation. This will be achieved by proposing novel network structures and an attention strategy. Second, multi-scale clinical knowledge is introduced to the hierarchical attention model by exploiting the clinical understanding of GI diseases together with a multi-scale feature fusion strategy. Then, we propose a joint semi-supervised and self-paced learning model for GI disease diagnosis, which is capable of utilizing unlabeled dataset and adaptively learning features from simple images to complex ones so as to maintain the robustness. Finally, we will evaluate and validate the proposed deep models and implement a graphical user interface to display visual diagnosis results.  Our preliminary experiments have demonstrated both the feasibility and the effectiveness of the proposed models. This project represents the first attempt to integrate comprehensive domain knowledge into deep learning models for GI disease diagnosis. It not only will produce high impacts in academic research of medical image analysis, but also can provide practical solutions to precision medicine.  


Project number9048179
Grant typeECS
Effective start/end date1/01/2112/12/22