On optimality of monotone channel-aware transmission policies: A constrained Markov Decision Process approach

A constrained Markov Decision Process (MDP) approach is deployed to prove the monotone structure of optimal channel-aware transmission policies for packet transmission over a correlated fading wireless channel subject to an average delay constraint. A transmission policy is a function mapping channel state information (CSI), buffer states and numbers of arriving packets to transmit probabilities. The objective is to minimize the average transmission energy cost subject to an average delay constraint. We use the Lagrange multiplier method to convert the constrained MDP to an unconstrained MDP and prove that the unconstrained optimal policy is threshold in the buffer state. It then follows that the constrained optimal transmission policy is a randomized mixture of two pure transmission policies that are threshold in the buffer occupancy.