Model-Free Optimal Tracking Design With Evolving Control Strategies via Q-Learning

This brief leverages a value-iteration-based Q-learning (VIQL) scheme to tackle optimal tracking problems for nonlinear nonaffine systems. The optimal policy is learned from measured data instead of a precise mathematical model. Furthermore, a novel criterion is proposed to determine the stability of the iterative policy based on measured data. The evolving control algorithm is developed to verify the proposed criterion by employing these stable policies for system control. The advantage of the early elimination of tracking errors is provided by this approach since various stable policies can be employed before obtaining the optimal strategy. Finally, the effectiveness of the developed algorithm is demonstrated by a simulation experiment.