Dual Timescales Coordinated and Optimal Voltages Control in Distribution Systems using Data-Driven and Physical Optimization

A large number of electric vehicles (EVs), distributed solar and/or wind turbine generators (WTGs) connected to distribution systems lead to frequent and sharp voltages fluctuations. The action rates of conventional adjustable devices and smart inverters are very different. In this context, a novel dual-timescale voltage control scheme is proposed by organically combining data-driven with physics-based optimization. On fast timescale, a quadratic programming (QP) for balanced and unbalanced distribution systems is developed based on branch flow equations. The optimal reactive power of renewable distributed generators (DGs) and static VAR compensators (SVCs) is configured on several minutes. Whereas, on slow timescale, a data-driven Markovian decision process (MDP) is developed, in which the charge/discharge power of energy storage systems (ESSs), statuses/ratios of switchable capacitors reactors (SCRs), and voltage regulators (VRs) are configured hourly to minimize long-term discounted squared voltages magnitudes deviations using an adapted deep deterministic policy gradient (DDPG) deep reinforcement learning (DRL) algorithm. The capabilities of the proposed method are validated with IEEE 33-bus balanced and 123-bus unbalanced distribution systems. The contributions of this paper are summarized as follows: (1)Combining data-driven with physics-based methods, a strategy for coordinated control of five different types of adjustable equipment, namely VRs, SCRs, ESSs, SVCs and DGs inverters on fast and slow timescales is proposed. (2) A slow timescale (say 1 hour) MDP for active and reactive power coordination is constructed. The (near) optimal settings of ratios/statuses of VRs, SCRs and charge/discharge power of ESSs are found using DRL algorithm. As a result, the deficiency of low computing rate for conventional physics -based large scale mixed integer non-convex nonlinear stochastic programming is completely overcome. (3) The charge/discharge power of ESS is continuous variable while ratios/statues of VRs and SCRs are discrete decisions. DDPG algorithm cannot be directly applicable to discrete action while DQN algorithm cannot be applicable to continuous action. Further, when there are a large number of VRs and SCRs in distribution network, DQN algorithm leads to dimensionality curses. The existing DRL algorithm in literatures cannot deal with joint continuous-discrete action (efficiently). To eliminate dimensionality curses in joint continuous- discrete action space, ratios/statuses of VRs and SCRs are firstly relaxed to continuous variables. Then, for the proto action given by actor of DDPG agent, Knn nearest neighbors are found out in the joint continuous-discrete action space. Finally, each of the Knn actions is transferred to the critic of DDPG agent one by one to evaluate its value. The action with the greatest value is chosen to interact with the distribution network. (4) Given the (near) optimal solution of the slow timescale MDP, a QP for VVO with DGs, SVCs inverters settings on fast timescale (say several minutes or seconds) to minimize squared voltages magnitudes deviations are developed for balanced and unbalanced distribution systems. As a result, voltage violations on fast timescale caused by sizable, rapid and frequent power fluctuations from renewable DGs and fast charged EVs can be mitigated in real time. (5) One of the outstanding advantages of the proposed method is very easy to perform in practice with near optimal solution. Further, when Knn =20 or Knn =40, the proposed method has much more stable training process than multi-agent DQN algorithm and much higher computing rate than conventional multi-slot single fast timescale mixed integer QP by about 18.0～36.7 times. © 2024 China Machine Press. All rights reserved.