Data-Driven Iterative Adaptive Dynamic Programming Algorithm for Approximate Optimal Control of Unknown Nonlinear Systems

In this paper, we develop a data-driven iterative adaptive dynamic programming algorithm to learn offline the approximate optimal control of unknown discrete-time nonlinear systems. We do not use a model network to identify the unknown system, but utilize the available offline data to learn the approximate optimal control directly. First, the data-driven iterative adaptive dynamic programming algorithm is presented with a convergence analysis. Then, the error bounds for this algorithm are provided considering the approximation errors of function approximation structures. To implement the developed algorithm, two neural networks are used to approximate the state-action value function and the control policy. Finally, two simulation examples are given to demonstrate the effectiveness of the developed algorithm.