DQN-Based Directional MAC Protocol in Wireless Ad Hoc Network in Internet of Things

The use of directional antennas in high-frequency bands (e.g., millimeter-wave) is essential to support applications requiring high throughput and low latency. However, communications using directional antennas require intricate scheduling by a central coordinator to avoid collision and deafness problems. Thus, in this study, we propose a directional medium access control (DMAC) protocol based on a deep $Q$ -network (DQN) framework wireless ad hoc networks (WANETs) for Internet of Things (IoT). In our model, even though there is no central coordinating unit (e.g., edge/cloud server), each IoT device can intelligently avoid the collision and deafness through its learning agent. In addition, to maximize the throughput, we design a reinforcement learning (RL) architecture and propose a DQN-based DMAC such that each IoT device intelligently selects the time-slot and transmitting beam without any central coordinator. The proposed schemes are evaluated using carrier-sense multiple access (CSMA) and adaptive learning-based DMAC (AL-DMAC) protocols. The evaluation results reveal that the proposed double DQN scheme outperforms the existing schemes by approximately 54.1% and 57.2% in terms of the throughput.