Sarsa-Based Output Tracking Problem of Stochastic Boolean Control Networks

In this paper, the output tracking problem (OTP) of stochastic Boolean control networks (SBCNs) is investigated by taking advantage of a reinforcement learning (RL) algorithm. Firstly, an algebraic expression of SBCNs is constructed based on the semi-tensor product (STP) of matrices. By working out states that can reach a given constant reference signal in one time step, the OTP is converted into a set stabilization problem (SSP). Secondly, a novel algorithm for exploring maximal control invariant subset (MCIS) is designed. To investigate the optimal control, the SSP is transformed into a path optimization problem (POP). Thirdly, a recursive algorithm of model-free RL, namely Sarsa, is adopted for resolving the POP. According to an optimal control sequence calculated by the Sarsa algorithm, an optimal state feedback controller is designed for resolving the SSP. Moreover, the application scope of Sarsa algorithm is expanded by introducing an undesirable set. Finally, a SBCN model of a biological network is adopted as an instance to illustrate the feasibility of the algorithms adopted.