Differentially Private and Communication-Efficient Federated Learning for AIoT: The Perspective of Denoising and Sparsification

As public awareness of privacy protection increases and data become more valuable, the applications of federated learning (FL) in the emerging field of Artificial Internet of Things (AIoT) has received widespread attention. Meanwhile, differential privacy (DP), providing strict privacy guarantees, has been introduced to meet users’ stringent privacy protection needs and increasingly sound laws and regulations. However, the implementations of DP in multiple iterations and rounds of FL training, as well as adding noise to all parameters without differentiation, will cause noise accumulation, resulting in the FL system to decline in performance or even fail to converge. To address the issue, FL with denoising DP and sparsification (DDPS-FL) is proposed in this article. First, a local denoising mechanism (LDM) suitable for DP with arbitrary noise adding mechanism is proposed. By removing the previously added noise from global models, LDM achieves direct noise reduction for clients. Second, sparsification based on parameter variation (SPV) is proposed to reduce noise indirectly by deleting nonsignificant parameters without compromising the level of privacy protection. Besides, SPV is able to multiple beneficial effects, such as saving privacy budget, stimulating the dynamism of FL training, amplifying privacy protection effect, and improving communication efficiency. Third, theoretical analysis is performed to prove that DDPS-FL can guarantee user privacy and has ideal convergence, and to analyze the impact of parameters, such as the number of training rounds and iterations on the system performance. Evaluation experiments based on four real-world datasets are elaborated to show that DDPS-FL outperforms state-of-the-art schemes in terms of training stability, model accuracy, and communication efficiency, and its performance becomes relatively better when more noise is added. © 2014 IEEE.