The Impact of Smoothness on Model Class Selection in Nonlinear System Identification: An Application of Derivatives in the RKHS

In this paper, we discuss the dependency between the kernel choice and the model class it represents. This is typically an undesired relationship, forcing the user to accept a trade-off between an acceptable variance characteristic and flexibility in the underlying function. Hence, a method is proposed in this paper that explicitly constrains the smoothness of the model, by regularizing over the derivative of the function. This not only broadens the available model class, but also simplifies the selection of any hyperparameters. We look at nonparametric models of nonlinear systems, and formulate the problem in the Reproducing Kernel Hilbert Space (RKHS). The proposed method is compared with an equivalent, established scheme by means of a simple example. It is shown that derivative-based regularization can help to extract useful structural information about an underlying system.