Understanding the Interplay Between Federated Learning and Decision-Making at Wireless Edge

Abstract

Intelligent devices at the wireless network edge, such as mobile phones and automobiles, generate a huge amount of data that can be exploited to enable smart applications. Collection and centralized storage of such data is time-consuming and expensive due to the lack of interconnection between different end devices, resulting in the widespread occurrence of data islands. Federated learning (FL) is a collaborative learning method tailored for isolated end devices. Data is retained locally without centralized collection, and only local model parameters are shared and aggregated. Decision-making is another critical issue at the wireless edge. From the device level, end devices need to perform some online decision-making tasks, such as edge caching and recommendation. From the system level, edge computing framework often needs to consider client scheduling, resource allocation, or their joint decisions to improve system-level benefits, like latency minimization. In this thesis, we point out that FL and decision-making are not isolated but closely linked. Through the analysis of the interplay between FL and decision-making problems, we propose corresponding algorithms and verify them on variant datasets, which in turn gain some insights to help us improve our proposed framework and algorithms.

First, the idea of FL can be utilized to collaborate in device-level decision-making, To be specific, we study the federated multi-armed bandit (FMAB) framework, in which each end device runs a MAB algorithm for online decision-making and we collaboratively utilize the knowledge of the multiple edge decision models. There are two increasing transmission-related concerns in FMAB, namely, data privacy and communication bottleneck. The privacy risk in FMAB is due to its intrinsic dependence on users’ feedback, which could leak users’ sensitive information. Meanwhile, the communication cost scales with the number of the end device, which is one of the primary bottlenecks of wireless FMAB. In this thesis, we design privacy-preserving mechanisms and efficient communication protocols for networked devices when performing online decision-making in three typical edge network structures, namely, centralized, decentralized, and hybrid structures. To be specific, we adopt the differential privacy (DP) approach to protect the user privacy at each device when exchanging information; we curtail communication costs by making less frequent communications with fewer agents’ participation. By deriving the learning regret of proposed algorithmic frameworks, we theoretically show trade-offs between regret and communication costs/privacy.

We address another two learning-related challenges of FMAB by integrating them into a multi-objective cache strategy design problem. A set of distributed cache nodes with storage space can pre-fetch some files from a remote central file server, where cache nodes have different preferences for the same set of files, and after caching a file, we can obtain a two-dimensional reward, namely cache hit and cache profit. In this thesis, we formulate this multi-objective caching problem as an FMAB with multi-dimensional rewards. We propose a two-step caching framework that estimates content profiles first and then optimizes cache placement. Specifically, to accommodate the reward heterogeneity, we utilize an adaptive federated learning-based estimation to approach the unknown content profiles. To address the multiple objective optimizations, we propose two effective methods to achieve adequate Pareto-optimal cache placement.

The system-level decisions can help us better deploy FL algorithms at the wireless edge. In particular, we focus on the decision-making problems in Hierarchical federated learning (HFL). In this thesis, we consider non-orthogonal multiple access (NOMA)-enabled HFL framework that provides a spectrum-efficient approach to enable massive client connectivity. Under this framework, we design novel client scheduling and resource allocation policies that decide on the subset of the devices to contribute at each round, and how the resources should be allocated among the participating devices, not only based on their wireless network conditions but also on the significance of their local data distribution. To be specific, we formulate the edge-client association problem as a many-to-one matching game with externalities and swap the connected pairs until the system utility no longer increases. For each temporary association result, we solve the CPU frequency and transmission power control problem for all end devices using a convex optimization method.

Date of Award	25 Apr 2023
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Linqi SONG (Supervisor)