Abstract
Deep learning algorithms have shown considerable potential in automating medical image diagnosis. A necessary condition to fully realize this potential is the construction of large-scale training datasets, which enables the algorithms to learn intrinsic diagnostic knowledge and provide trustworthy diagnosis. Unfortunately, in realistic medical scenarios, the data from a single hospital is typically inadequate to train a diagnostic model with strong generalization capabilities, while centralizing data from different hospitals to construct large-scale datasets is impractical due to privacy concerns. Federated learning (FL), a decentralized learning paradigm, offers a feasible solution to this dilemma, which learns a global model by sharing the model parameters among clients without exchanging private data, thereby preserving data privacy.Despite its promise, applying FL to train diagnosis models inevitably suffers from some critical challenges in real-world medical scenarios as follows. (1) Data scarcity: Hospitals frequently face data insufficiency due to low incidence rates of certain diseases, limited resources for data collection, or stringent data protection policies. (2) Annotation scarcity: Pixel/voxel-level annotations might be unavailable for hospitals since the acquisition of dense annotations is extremely time-consuming and labor-intensive, and highly depends on specialized clinicians. (3) Data heterogeneity: Private data from different sites exhibit significant inter-site heterogeneity arising from variations in imaging protocols and population demographics. (4) Diagnosis security: Diagnostic models may fail to recognize unknown diseases. These diseases probably emerge without warning and will be misclassified into known diseases, leading to misdiagnosis. These challenges result in poor generalization of diagnostic models, severely hindering the development of trustworthy medical diagnosis. In this thesis, I present a series of feasible solutions to handle these challenges, aiming to achieve the objective -accurate, secure and privacy-preserving medical intelligent diagnosis.
First, finetuning foundation models using low-rank adaptation (LoRA) is a common way to achieve data-efficient training when the data of hospitals is limited. Nevertheless, this strategy undergoes aggregation deviation and noise amplification effect in the FL setting. To overcome these issues, I propose a novel privacy-preserving federated finetuning framework called Deviation Eliminating and noise Regulating (DEeR). In this framework, a deviation eliminator is designed to utilize an alternating minimization algorithm to iteratively optimize the zero-initialized and non-zero-initialized parameter matrices of LoRA, ensuring that aggregation deviation remains zero throughout the training process. To suppress the noise amplification effect, I propose a noise regulator that exploits two regulator factors to decouple the relationship between differential privacy and LoRA, thereby achieving robust privacy protection and excellent finetuning performance.
Second, to avoid the need for pixel-level annotations, I propose a federated weakly supervised segmentation framework -- Federated Drift Mitigation (FedDM) -- under the supervision of bounding boxes. This framework consists of Collaborative Annotation Calibration (CAC) and Hierarchical Gradient De-conflicting (HGD). CAC customizes a distal peer client and a proximal peer client for each client, then leverages inter-client knowledge agreement and disagreement to recognize clean labels and correct noisy labels hidden in weak annotations. Moreover, HGD builds a client hierarchy online during each communication round. Through de-conflicting clients under the same parent nodes from bottom layers to top layers, HGD ensures robust gradient aggregation at the server side.
Third, I propose a framework named Federated Bias eliMinating (FedBM) to eliminate local learning bias in classifier and feature extractor caused by data heterogeneity. FedBM possesses two modules, i.e., Linguistic Knowledge-based Classifier Construction (LKCC) and Concept-guided Global Distribution Estimation (CGDE). To mitigate the classifier bias, LKCC exploits class concepts, prompts, and pre-trained language models (PLMs) to obtain concept embedding distributions. These distributions are then used to pre-construct a high-quality classifier for clients. To alleviate the feature extractor bias, CGDE samples probabilistic embeddings from the distributions to train a conditional generator. The generator is shared by all clients and produces pseudo data to calibrate updates of local feature extractors.
Lastly, to achieve unknown disease recognition, I propose a Federated Open Set Synthesis (FedOSS) framework to generate virtual unknown samples. FedOSS consists of two modules: Discrete Unknown Sample Synthesis (DUSS) and Federated Open Space Sampling (FOSS). DUSS exploits inter-client knowledge inconsistency to recognize known boundary samples, which are then converted into virtual unknown samples via model inversion. FOSS unites these generated unknown samples from different clients to estimate the class-conditional distributions of open data space near decision boundaries and further samples more open data, thereby enhancing the diversity of virtual unknown samples.
In summary, the proposed methods significantly advances medical intelligent diagnosis by addressing key challenges. FedDM and FedBM are tailored to optimize lesion localization and classification performance, respectively. DEeR alleviates data scarcity during the training of FedDM and FedBM, while FedOSS extends FedBM’s capability to detect unknown diseases. Through synergistic integration, these approaches improve diagnostic accuracy, security, and patient privacy, ensuring trustworthy clinical deployment. Extensive validation across diverse medical imaging modalities - including histopathology, endoscopy, dermoscopy, OCT, microscopy, CT, and MRI - demonstrates consistent superiority over state-of-the-art methods, confirming clinical reliability and adaptability of these methods.
| Date of Award | 29 Jul 2025 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Shijun ZHAO (Supervisor), Jun Liu (External Co-Supervisor) & Yixuan Yuan (External Co-Supervisor) |