Service-aware Q-learning-based routing protocol in the underwater acoustic sensor network

Due to the high bit error ratio, long propagation delay, low transmission bandwidth and limited energy of sensor nodes, designing an appropriate routing protocol is an important issue in the underwater acoustic sensor network (UASN). In this paper, a service-aware Q-learning-based routing (SAQR) protocol is proposed in the UASN to support the diverse services, as well as achieving an efficient routing. The proposed protocol incorporates four factors in the reward function, and selects the neighboring node with the minimum Q value as the next hop. In addition, a set of candidate forwarding nodes is selected from the neighboring nodes, and a holding time mechanism is addressed to facilitate opportunistic transmission and improve the packet delivery ratio (PDR). Furthermore, a routing void handling mechanism is developed to enhance the reliability of data transmission in the UASN. Simulation results show that the proposed SAQR protocol performs well, in terms of the PDR, the end- to-end delay, the energy efficiency, in the dynamic underwater network environment.