Disentangle Then Calibrate: Selective Treasure Sharing for Generalized Rare Disease Diagnosis

Annotated images for rare disease diagnosis are extremely hard to collect. Therefore, identifying rare diseases based on scarce amount of data is of far-reaching significance. Existing methods target only at rare diseases diagnosis, while neglect to preserve the performance of common disease diagnosis. To address this issue, we first disentangle the features of common diseases into a disease-shared part and a disease-specific part, and then employ the disease-shared features alone to enrich rare-disease features, without interfering the discriminability of common diseases. In this paper, we propose a new setting, i.e., generalized rare disease diagnosis to simultaneously diagnose common and rare diseases. A novel selective treasure sharing (STS) framework is devised under this setting, which consists of a gradient-induced disentanglement (GID) module and a distribution-targeted calibration (DTC) module. The GID module disentangles the common-disease features into disease-shared channels and disease-specific channels based on the gradient agreement across different diseases. Then, the DTC module employs only disease-shared channels to enrich rare-disease features via distribution calibration. Hence, abundant rare-disease features are generated to alleviate model overfitting and ensure a more accurate decision boundary. Extensive experiments conducted on two medical image classification datasets demonstrate the superior performance of the proposed STS framework.