Online Offloading in Dense Wireless Networks: An Adversary Multi-armed Bandit Approach

In this paper, we consider the offloading problem in the dense wireless networks, whereby each user adaptively assigns its flows to different neighboring access points to maximize the utility. We assume that the channel state information is unknown to the users, which raises two challenging issues. Firstly, the performance achieved from each access point is unknown and has to be learned sequentially due to dynamic channel condition. Secondly, the achieved performance also depends on the set of competing flows, which is unpredictable since the decisions of other users are made online. We formulate this problem as an online learning problem based on the adversarial multi-armed bandit framework, which accommodates both the arbitrary variation of channel condition and the unpredictable competition among flows. An online offloading algorithm is presented based on an exponentially weighted average strategy, which is shown to converge to a set of correlated equilibria with vanishing regrets through theoretical analysis and simulations.