Deep Reinforcement Learning for Distributed Coordinated Beamforming in Massive MIMO

In this paper, we investigate a dynamic coordinated beamforming (CBF) problem to enhance the sum rate of a massive multiple-input multiple-output (MIMO) cellular network. Although existing optimization-based algorithms can provide near-optimal solutions, they require real-time global channel state information (CSI) and have high computational complexity, making them not viable in practical mobile networks. To tackle this issue, we propose a deep reinforcement learning based distributed dynamic CBF framework, which allows each base station (BS) to determine the optimal beamformers with only local CSI and some historical information transferred from other BSs. Besides, the computational complexity is substantially reduced thanks to the exploitation of neural networks and expert knowledge, i.e., a known solution structure that can be observed from a closed-form optimization algorithm. Simulation results demonstrate that the proposed approach can outperform the closed-form optimization methods and achieve comparable performance to the state-of-the-art optimization algorithm.