Hierarchical Charging and Computation Scheduling for Connected Electric Vehicles via Safe Reinforcement Learning

By utilizing the energy storage capabilities of connected electric vehicle (CEV) batteries and their computing capacities, grid-powered vehicular networks can facilitate the integration of renewable energy sources and offer energy-efficient computational service provisioning. However, the uncertain electricity/service demand and the grid's physical safety constraints hinder effective CEV resource scheduling. In this paper, we formulate a hierarchical electricity and computing resources scheduling problem for CEVs in a microgrid under power flow constraints and further transform it into a constrained Markov decision process. By combining safe reinforcement learning with a linear programming module, we present a reinforcement-learning-based Safe Hierarchical Charging and Computation Scheduling (SHCCS) scheme that operates at two levels: the upper level focuses on large-scale power distribution scheduling for CEV charging stations, while the lower level handles small-scale charging and computation scheduling for individual CEVs. Through extensive simulations conducted on the IEEE 33-bus test system, we demonstrate that our scheme improves power system operating revenue while ensuring grid safety.