Dantzig-Selector Radial Basis Function Learning with Nonconvex Refinement

This paper addresses the problem of constructing nonlinear relationships in complex time-dependent data. We present an approach for learning nonlinear mappings that combines convex optimization for the model order selection problem followed by non-convex optimization for model refinement. This approach exploits the linear system that arises with radial basis function approximations. The first phase of the learning employs the Dantzig-Selector convex optimization problem to determine the number and candidate locations of the RBFs. At this preliminary stage maintaining the supervised learning relationships is not part of the objective function but acts as a constraint in the optimization problem. The model refinement phase is a non-convex optimization problem the goal of which is to optimize the shape and location parameters of the skew RBFs. We demonstrate the algorithm on on the Mackey-Glass chaotic time-series where we explore time-delay embedding models in both three and four dimensions. We observe that the initial centers obtained by the Dantzig-Selector provide favorable initial conditions for the non-convex refinement problem.