Cross-regional path planning based on improved Q-learning with dynamic exploration factor and heuristic reward value

This paper proposes a cross-regional path planning method based on improved QL, whose enhancements focus on two aspects. One is to dynamically adjust the exploration factor in the e-greedy strategy, guided by the principle of SA. This helps prevent unstable action selection or falling into local optima. The other is to incorporate the Euclidean distance between agent and the target point as heuristic information to smooth the reward values, thereby reducing the blind search in path exploration. Simulation experiments are conducted in two different scenarios to validate the effectiveness and adaptability of improved QL. Results from Experiment 1 demonstrate that compared to GA, PSO, SARSA and QL, the enhanced algorithm can plan the optimal path in the shortest duration. Experiment 2, which introduces congested road conditions, proves that proposed QL has advantages over original QL in complex environments in terms of search cost, operational efficiency, and convergence speed. This study extends the research of RL in the field of path planning to the cross-regional scope. It provides algorithmic support for the development of new cross-regional transport systems such as ICV.