Shrinkage estimation of higher-order Bochner integrals

We consider shrinkage estimation of higher-order Hilbert space-valued Bochner integrals in a non-parametric setting. We propose estimators that shrink the U-statistic estimator of the Bochner integral towards a pre-specified target element in the Hilbert space. Depending on the degeneracy of the kernel of the U-statistic, we construct consistent shrinkage estimators and develop oracle inequalities comparing the risks of the U-statistic estimator and its shrinkage version. Surprisingly, we show that the shrinkage estimator designed by assuming complete degeneracy of the kernel of the U-statistic is a consistent estimator even when the kernel is not completely degenerate. This work subsumes and improves upon Muandet et al. (J. Mach. Learn. Res. 17 (2016) 48) and Thou, Chen and Huang (J. Multivariate Anal. 169 (2019) 166-178), which only handle mean element and covariance operator estimation in a reproducing kernel Hilbert space. We also specialize our results to normal mean estimation and show that for d >= 3, the proposed estimator strictly improves upon the sample mean in terms of the mean squared error.