Privacy-Preserving Deep Learning for Medical Image Analysis under Data Scarcity

Abstract

Medical imaging is an important diagnostic tool for various diseases, such as functional magnetic resonance imaging (fMRI) for mental disorder diagnosis and X-ray images for Covid-19/pneumonia diagnosis. However, prolonged and repetitive image analysis tasks can lead to fatigue and burnout among medical professionals, affecting concentration and potentially compromising the accuracy of interpretations. Recently, deep learning-based computer-aided diagnosis (DL-CAD) models have achieved powerful capabilities of assisting doctors in interpreting medical images.

However, training DL-CAD models with satisfied performance usually faces two challenges, i.e., data scarcity and privacy of medical images. Due to the high confidentiality and sensitivity of medical images, it is impractical to increase the amount of available training samples via data sharing strategy. That is to say, the training of DL-CAD models within single healthcare institutions usually tends to face the dilemma of data scarcity, resulting in the overfitting problem. This thesis focuses on developing privacy-preserving deep learning models to address the data scarcity problem by avoiding explicit high-sensitivity medical sample transfer. Specifically, we ease the data scarcity mainly from the perspectives of multi-institutional collaboration and transfer learning from natural images. The multi-institutional collaboration addresses data scarcity by leveraging inter-domain knowledge, which can be hampered by data privacy. Furthermore, the data distribution divergence among various institutions may also pose a challenge in using external knowledge. As for knowledge transfer from non-private natural images, it aims to unleash the ontological potential of visual images and enhance representation learning of DL-CAD models. The distinct cross-dataset distribution heterogeneity, however, degrades the effectiveness of model training.

Firstly, this thesis presents a comprehensive survey of diverse methods to address data scarcity from the perspective of multi-institutional collaboration and knowledge transfer from natural images. Moreover, the challenges and limitations in current studies, especially in the clinical applications, are elucidated.

Secondly, three privacy-preserving multi-institutional collaboration methods and one cross-dataset adaptation approach are proposed to ease medical data scarcity. Specifically, from the view of multi-party collaboration, a federated multi-task learning framework is proposed for the joint diagnosis of multiple mental disorders based on fMRI samples. This framework eases data scarcity by simultaneously using fMRI scattered in different institutions and exploiting significantly overlapping symptoms between these disorders. Despite the remarkable results achieved, our framework assumes that all institutions have fully-labeled samples to train on, which is contrary to real-world scenarios. Under privacy protection, we further propose a source-free semi-supervised transfer learning method to leverage the unlabeled fMRI samples and inter-institutional knowledge. The simulation experiments demonstrate that the source-free transfer nature and exploitation of unlabeled samples enhance the diagnostic performance without explicitly transferring raw samples. However, the decent results still rely on similar source domains with identical data modalities. To ease the dependence, we propose a cross-dataset adaptation approach to transfer knowledge from natural images to X-rays. Corresponding results demonstrate that easing the domain shifts between X-rays and the selected natural images improves the generalization ability of CAD models. Based on this method, we propose a heterogeneous structured federated learning framework to account for the need to personalize models, using natural images as an intermediate for heterogeneous structured model aggregation to avoid uploading local private samples.

Lastly, on several real-world datasets, the proposed methods have been demonstrated to effectively handle medical data scarcity problems by promoting privacy-preserving multi-institutional collaboration, easing the cross-domain gap between natural images and X-rays, and meeting the personalization requirements. Furthermore, the frameworks also provide interpretable results to capture the key regions of interest of different modality medical images, which can be used to aid in understanding pathological progress and clinical decision-making.

Date of Award	26 Aug 2024
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Linqi SONG (Supervisor) & Kay Chen Tan (External Co-Supervisor)