Statistical Properties of Adversarial Training in the Reproducing Kernel Hilbert Space

While machine learning models and artificial intelligence systems have grown increasingly powerful, researchers have observed that even small perturbations in input data can lead to significantly unreliable outputs. To address this issue, adversarial training has been proposed as a method to improve the model robustness by optimizing the worst-case loss under bounded input perturbations. While extensive research has focused on adversarially trained parametric models, adversarial training in nonparametric models, which are capable of capturing more complex data structures, has not been well understood. This project aims to investigate adversarially trained nonparametric models within the framework of Reproducing Kernel Hilbert Space (RKHS).The resulting problem is a min-max optimization problem, which can be considered as a new statistical estimation problem within the RKHS framework. The research question is to analyze the statistical properties of the associated estimates. The first task is to characterize the asymptotic behavior of the adversarially trained RKHS estimates, based on which the statistical inferences can be developed accordingly. The second task will focus on the finite-sample convergence rate of the generalization error of the estimates and evaluate the conditions for which the statistical optimality can be proven. To facilitate more efficient applications of the nonparametric adversarial training, possible improvements will be explored to strike a tunable balance between the adversarial robustness and estimation accuracy in the RKHS. Finally, to make our analysis applicable for broader topics, extension to more general robust optimization will be considered.By delivering the tasks mentioned above, both the asymptotic and non-asymptotic behavior of adversarial training estimates in the RKHS will be characterized. Then, the bias induced by adversarial robustness can be analytically evaluated through asymptotic analysis. Also, the finite-sample analysis helps us identify the conditions under which the estimates can be statistically optimal, i.e., the accuracy is not sacrificed for robustness. Together with explorations of the trade-off techniques, these insights contribute to a deeper depiction and solution of the central challenge in adversarial training—the trade-off between adversarial robustness and estimation accuracy—in the RKHS scheme.The main challenge in this project lies in analyzing the new framework of the RKHS-based min-max optimization problem. The challenges can be addressed by bridging the work on parametric min-max optimization problems, including the P.I.'s previous research, and the classical RKHS-based empirical risk minimization framework.