A model-based Q-learning scheme for wireless channel allocation with prioritized handoff

In this paper we propose a new channel allocation scheme for improving the quality of service in cellular mobile networks. The proposed algorithm prioritizes handoff call requests over new call requests. The goal is to reduce the handoff failures while still making efficient use of the network resources. A performance measure is formed as a weighted linear function of new call and handoff call blocking probabilities. This problem is formulated as a semi-Markov decision process with an average cost criterion. A simulation-based learning algorithm is then developed to approximate the optimal control policy online using the generated samples from direct interactions with the network. It is based on an approximate model that is estimated simultaneously while learning a control policy. The estimated model is used to direct the search for an optimum policy. Extensive simulations are provided to assess the effectiveness of the algorithm under a variety of traffic conditions. Comparisons with some well-known allocation policies are also presented. Simulation results show that for the traffic conditions considered in this paper, the proposed scheme has a comparable performance to the optimal guard channel approach.