Abstract
While deep learning has exhibited exceptional performance across a broad range of real-world applications, the theoretical underpinnings that explain its efficacy remain largely elusive. This thesis delves into the theoretical analysis of deep learning methodologies, with neural networks, we explore their capabilities in approximating functions with mixed smoothness and functionals attaching to a periodic Korobov space, and addressing high-dimensional machine learning challenges. A brief description of the organization of this thesis is as follows.Chapter 1 provides an overview of the historical development of machine learning, establishing the motivation and basic definitions for investigating the theory of function approximation within the context of neural networks in this thesis.
Chapter 2 presents a work on the approximation capabilities of shallow neural networks when applied to functions of mixed smoothness. A dimensionally independent approximation rate is established. A regression learning algorithm is also developed, and a dimension-independent learning rate is derived.
Chapter 3 generates the dimension-independent approximation rates for approximating nonlinear continuous functions in the Korobov Space with Lp norm. Following the result, a natural generalization analysis is discussed. Furthermore, a classification algorithm based on Tikhonov regularization is discussed. The learning rates for the excess misclassification error using the convex η-norm loss function are derived. Furthermore, the error estimates under Tsybakov noise conditions are presented.
Chapter 4 explores functional approximation for functions chosen from the weighted Korobov space. Our study established theoretical results on the approximation capabilities of specialized FNN architectures, which consist of FNO layers followed by a multichannel DCNN. We focused on high-dimensional functions, deriving key results such as error bounds and expressivity properties of deep networks. These findings emphasize the efficiency of deep networks in representing complex functions with fewer parameters.
Chapter 5 contains an ongoing work that investigates the generalization analysis of approximating Korobov functions using deep Fourier Neural Operators(FNOs). The core of this analysis involves a novel equivalent definition that enables probability measurements on the filter operator, including FIR kernels, truncated
Fourier coefficients and their inversions.
Chapter 6 exhibits another ongoing research where the structure of periodic CNN is studied and its capability to approximate the ridge function is proposed. The next step is to discuss its capability of approximation in high dimensions.
Finally, chapter 7 serves as a conclusion section that wraps up all the above research experience and provides the blueprint for the future research plan.
| Date of Award | 23 Jun 2025 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Dingxuan ZHOU (Supervisor) & Xiang ZHOU (Supervisor) |