Monte Carlo TD(λ)-methods for the optimal control of discrete-time Markovian jump linear systems

In this paper we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discrete-time Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. It is well known that the optimal control of this problem is given in terms of the maximal solution of a set of coupled algebraic Riccati equations (CARE), which have been extensively studied over the last few years. We trace a parallel with the theory of TD(lambda) algorithms for Markovian decision processes to develop a TD(lambda) like algorithm for the optimal control associated to the maximal solution of the CARE. Some numerical examples are also presented.