Generic Frameworks for Distributed Functional Optimization and Learning Over Time-Varying Networks

In this paper, we establish a distributed functional optimization (DFO) theory over time-varying networks. The vast majority of existing distributed optimization theories are developed based on Euclidean decision variables. However, for many scenarios in machine learning and statistical learning, such as reproducing kernel spaces or probability measure spaces where functions or probability measures serve as the fundamental variables, the development of existing distributed optimization theories exhibit obvious theoretical and technical deficiencies. To address these issues, we develop a novel general DFO theory on Banach spaces, allowing functional learning problems in the aforementioned scenarios to be incorporated into our framework for resolution. We study both convex and nonconvex DFO problems. We propose a distributed functional mirror descent algorithm and a distributed functional gradient descent algorithm to solve them. Comprehensive convergence theory of these methods is rigorously established and satisfactory convergence rates are fully derived. The work provides generic analytical frameworks for DFO. The established theory is shown to have crucial application value in kernel-based distributed learning theory over time-varying networks. © 2014 IEEE.