Fault diagnosis and protection strategy based on spatio-temporal multi-agent reinforcement learning for active distribution system using phasor measurement units

Active distribution system (ADS) requires intelligent sensors to provide real-time data. Due to the harmonic distortion and sparse reward function, the multi-agent reinforcement learning strategy has the fuzzy characteristic and slow convergence. This work proposes a model-free spatio-temporal multi-agent reinforcement learning (STMARL) strategy for the spatio-temporal fault diagnosis and protection. The augmented-state extended Kalman filter tracks spatial-temporal sequences measured by phasor measurement unit (PMU) and feed into the diagnosis model. The supervised multi-residual generation learning (SMGL) model is constructed to diagnose the single-phase-to-ground fault. Based on spatio-temporal sequences, the SMGL diagnosis model integrates the ADS protection as a Markov decision process and the protection operation is quantified as the STMARL reward. In the hybrid multi-agent framework, the STMARL protection strategy converges faster based on the higher-level agent suggestion without the global reward. The STMARL protection strategy is validated in the IEEE 34-bus distribution test system with 10 PMUs. Comparing with the SOGI, WNN, Sarsa and DDPG algorithms, in the common fault conditions, the STMARL protection strategy shows better performance in the high dynamic environment with the response time 1.274 s and the diagnosis accuracy rate 97.125%. The STMARL diagnosis and protection strategy guides ADS in a stable operation coordinate with all PMUs, which lays foundation for the synchronous measurement application in the smart grid.