Nonlinear black-box models in system identification: Mathematical foundations

We discuss several aspects of the mathematical foundations of the nonlinear black-box identification problem. We shall see that the quality of the identification procedure is always a result of a certain trade-off between the expressive power of the model we try to identify (the larger the number of parameters used to describe the model, the more flexible is the approximation), and the stochastic error (which is proportional to the number of parameters). A consequence of this trade-off is the simple fact that a good approximation technique can be the basis of a good identification algorithm. From this point of view, we consider different approximation methods, and pay special attention to spatially adaptive approximants. We introduce wavelet and 'neuron' approximations, and show that they are spatially adaptive. Then we apply the acquired approximation experience to estimation problems. Finally, we consider some implications of these theoretical developments for the practically implemented versions of the 'spatially adaptive' algorithms.