Improved rates and asymptotic normality for nonparametric neural network estimators

In this paper, we obtain an improved approximation rate tin Sobolev norm) of r(-1/2-alpha/(d+1)) for a large class of single hidden layer feedforward artificial neural networks (ANN) with r hidden units and possibly nonsigmoid activation functions when the target function satisfies certain smoothness conditions, Here, d is the dimension of the domain of the target function, and alpha is an element of (0, 1] is related to the smoothness of the activation function. When applying this class of ANN's to nonparametrically estimate (train) a general target function using the method of sieves, we obtain new root-mean-square convergence rates of O-P([n/log (n)](-(1+2 alpha/(d+1))/[4(1+alpha(d+1))])) = O-P(n(-1/4)) by letting the number of hidden units r, increase appropriately with the sample size (number of training examples) n, These rates are valid for i.i.d. data as well as for uniform mixing and absolutely regular (beta-mixing) stationary time series data. In addition, the rates are fast enough to deliver root-n asymptotic normality for plug-in estimates of smooth functionals using general ANN sieve estimators, As interesting applications to nonlinear time series, we establish rates for ANN sieve estimators of four different multivariate target functions: a conditional mean, a conditional quantile, a joint density, and a conditional density. We also obtain root-n asymptotic normality results for semiparametric model coefficient and average derivative estimators.