Kernel-based Adaptive Huber Mean Regression with Heavy Tails and Contamination

We consider robust conditional mean learning in the mathematical framework of reproducing kernel Hilbert spaces, assuming the errors can be heavy-tailed and at the same time some responses are subject to arbitrary contamination. Simultaneous robustness to both heavy tails and contamination are of interest. When there is no contamination, we establish rates when the (1 + θ)-moment (θ > 0) of the error exists, which matches the minimax rate for the least squares regression as soon as θ ≥ 1, although the existing upper bounds for the least squares regression usually assume sub-Gaussian or sub-exponential error distribution. The existence of contamination adds an additional term to the bound. We also consider learning using random features, which is a computationally efficient kernel approximation method to relieve the computational burden of kernel-based learning. We show the same rates can be achieved with the number of random features much smaller than the sample size, extending previous results on random features for least squares kernel regression. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. © 2026 The Author(s). Published with license by Taylor & Francis Group, LLC.