Personalized federated learning with multiple classifier aggregation

Personalized federated learning (PFL) has garnered attention due to its capability to address statistical heterogeneity among clients. Typically, prevailing PFL methods aggregate a single global model for personalization, which may be inadequate for clients with diverse data distributions. Furthermore, in the local update, each private dataset is used to optimize the model independently, which increases the risk of overfitting the current data distribution and losing previously acquired knowledge, resulting in knowledge forgetting. In this study, a personalized federated learning with multiple classifier aggregation (FedMCA) method is proposed. FedMCA splits the client model into its head and base, optimizing them respectively using an alternating strategy that sequentially targets the head and base. Initially, to address the suboptimal model problem, the proposed method aggregates multiple classifiers using data distribution and employs knowledge distillation to impart positive and negative classifier knowledge for learning the most suitable personalized model head. Additionally, to mitigate knowledge forgetting, a learnable personalization layer is introduced, and hidden loss is utilized to learn the knowledge of the global base and prevent overfitting of the model base. The experimental results demonstrate that the proposed method achieves competitive performance across various benchmarks, outperforming most state-of-the-art PFL algorithms. The source code is publicly available at https://github.com/xiaye-maker/FedMCA. © 2025 Elsevier B.V.