Autonomous Delay Tolerant Network Management Using Reinforcement Learning

Delay tolerant networks (DTNs) offer a set of standardized protocols to enable Internet-like connectivity across the solar system. Unlike other protocols such as the Transmission Control Protocol (TCP) and the Internet Protocol (IP), DTN protocols are robust to end-to-end connection disruptions and long delays. Although the behavior of DTN core protocols is well understood, management of DTNs is still an area of active research. This paper uses reinforcement learning (RL) to automate the management of a DTN node consisting of an orbital relay between the moon and Earth. More specifically, the RL agent is in charge of deciding when to drop packets, when to change the data rate of the neighbor node links, when to reroute bundles to crosslinks, or when not to change any network parameter. The agent's goal is to maximize the bits received by the Deep Space Network while minimizing the capacity allocated to all controlled links, and control the buffer utilization to avoid memory overflows. To assess the potential of using RL in DTN management, the performance of the trained RL agent is benchmarked against other non-RL-based policies in a realistic lunar scenario. Results show that the RL agent provides the highest reward, outperforming all non-RL policies in this scenario.