Markov decision processes with random horizon

In this paper we formulate Markov Decision Processes with Random Horizon (MDPRH). We show the optimality equation for the MDPRH, however there may not exist optimal stationary strategies, or E-optimal stationary strategies for the processes. When the MDPRH has the probability distribution for the planning horizon with infinite support, we show Turnpike Planning Horizon Theorem. Then we evaluate rolling strategies and develop an algorithm obtaining an optimal first stage decision. Finally, some numerical experiments on a simple inventory model are done to understand the phenomena.