Neuro-dynamic programming for task allocation to unmanned aerial vehicles

In this paper, we propose a model for allocating tasks to unmanned aerial vehicles (UAVs) in a dynamic environment. The problem is naturally formulated as a dynamic programming problem, which is hard to calculate and time consuming. We use a Neuro-Dynamic Programming (NDP) framework to allocate emergent tasks to UAVs, where the optimal reward function is approximated using a neural network trained on simulated data. A heuristic-based formula is used to formulate the reward of assignment based on features extracted from the state space of the problem. We report results on computation time and average path length of the solution for several different numbers of problem sizes. The NDP model outperformed a well-accepted heuristic-based network optimization model (ICTP) in terms of time to generate the solution. The quality of solution produced by the NDP model is comparable, or sometimes even better than that of ICTP.