UNBIASED ESTIMATE OF GENERALIZATION ERROR AND MODEL SELECTION IN NEURAL-NETWORK

Model selection is based upon the generalization errors of the models in consideration. To estimate the generalization error of a model from the training data, the method of cross-validation and the asymptotic form of the jackknife estimator are used. The average of the predictive errors is used to estimate the generalization error. This estimate is also used as the model selection criterion. The asymptotic form of this estimate is obtained. Asymptotic model selection criterion is also provided for the case when the error function is the penalized negative log-likelihood. In the regression case, it also proves the asymptotic equivalence of Moody's model selection criterion and the cross-validation method under a condition on the error function.