Model-Free Optimal Stabilization of Unknown Time Delay Systems Using Adaptive Dynamic Programming

This paper deals with the model-free optimal stabilization problem of time delay systems with unknown delays and unknown system dynamics using adaptive dynamic programming (ADP). We consider a general linear system with multiple state and input delays. An extended state augmentation approach is presented that brings the system into a delay-free form without requiring the knowledge of the delays. As both the number of delays and the lengths of the delays are unknown, we utilize upper bounds of the state and input delays to augment the original state vector with delayed inputs and states. Controllability conditions of the extended augmented system are established. An iterative Q-learning scheme is employed to learn the optimal control parameters based on the online data. In addition to handling the unknown delays, the proposed scheme also uplifts the requirement of the bicausal change of coordinates, as found in the previous ADP methods for time delay problems. Simulation results are presented to demonstrate the effectiveness of the proposed scheme.