Optimality of monotone policies for transmission control with switching costs

This paper considers a time slotted transmission system over a finite state Markovian channel where in each time slot a transmission mode is dynamically selected from a finite set. There is a switching cost for changing the transmission mode and there is a channel dependent delay cost associated with each transmission mode. We formulate the optimal trade-off between the transmission latency (delay cost) and the mode switching cost as an infinite horizon Markov Decision Problem (MDP). By exploiting special structures of the formulated MDP and under certain sufficient conditions, we show that optimal transmission mode selection policies are monotone in the state variables. Furthermore, the transmission control exhibits more resistance to change the transmission mode as compared with the case with no switching cost. These monotone (threshold-based) structures allow efficient numerical solutions along with an improved qualitative assessment of optimal transmission control rules.