Cooperative optimal scheduling strategy of source and storage in microgrid based on soft actor-critic

被引：0
|
作者：
Liu L. [1 ]
Zhu J. [1 ]
Chen J. [1 ]
Ye H. [1 ]
机构：
[1] School of Electric Power, South China University of Technology, Guangzhou
来源：
Dianli Zidonghua Shebei/Electric Power Automation Equipment | 2022年 / 42卷 / 01期
基金：
中国国家自然科学基金;
关键词：
Deep learning; Distributionally robust optimizationKey words:microgrid; Energy storage; Generation and transmission system; Optimization and planning; Reinforcement learning; Renewable energy; Robust optimization; Security constraint; Soft actor-critic; Uncertainty;
D O I：
10.16081/j.epae.202110036
中图分类号：
学科分类号：
摘要：
In recent years, the proportion of renewable energy and energy storage in microgrid is increasing, which brings new challenges to its optimal scheduling. Aiming at the difficulty in solving the cooperative optimal scheduling problem of source and storage in microgrid due to the non-convex nonlinear constraints, the deep reinforcement learning algorithm is used to construct the data-based strategy function, and the optimal strategy is found out through continuous interactive learning with the environment, so that avoiding the direct solution of the original non-convex nonlinear problem. Considering the strategy function may not meet the security constraints in the training process, furthermore, a learning method of cooperative optimal scheduling secure strategy of source and storage in microgrid based on partial model information is proposed, and the optimal strategy meeting the network security constraints is obtained. In addition, aiming at the problem of long time-consuming due to the interaction between agents and environment in the training process for reinforcement learning, the neural network is used to model the environment, so as to improve the learning efficiency.Review and prospect of robust optimization and planning research on generation and transmission systemYUAN　Yang1, ZHANG　Heng1, CHENG　Haozhong1, LIU　Lu1, ZHANG　Xiaohu2, LI　Gang2, ZHANG　Jianping2(1. Key Laboratory Control of Power Transmission and Conversion, Ministry of Education, Shanghai Jiao Tong University, Shanghai 200240, China;2. East China Branch of State Grid Corporation of China, Shanghai 200120, China)With the uncertainty of power system increasing gradually, the application of robust optimization and planning research on generation and transmission system to resist the uncertainty of extremely scenes has become a significant research method. Firstly, the robust optimization is divided into classical robust optimization and distributionally robust optimization from the perspective of whether the probability distribution characteristics of uncertain factors are considered, the mathematical models and uncertain set characteristics of these two kinds of robust optimization are sorted out. Secondly, the existing classical robust optimization and distributionally robust optimization research on generation and transmission system are divided into three aspects:considering the uncertainty of node injection power, considering the uncertainty of power capacity growth and cost, and considering the uncertainty of transmission network state, and the research framework and limitations of robust optimization planning research on generation and transmission system are refined. Finally, the problems worthy of further study in robust optimization planning on power generation and transmission system are prospected, which provides ideas and directions for the robust optimization planning follow-up research on power generation and transmission system. © 2022, Electric Power Automation Equipment Press. All right reserved.
引用
收藏
页码：79 / 85
页数：6
相关论文
共 19 条
  • [11] CHENG R, OROSZ G, MURRAY R M, Et al., End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks, Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3-6, (2019)
  • [12] ELFWING S, SEYMOUR B., Parallel reward and punishment control in humans and robots: safe reinforcement learning using the MaxPain algorithm, 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, pp. 140-147, (2017)
  • [13] DJORDJE G, SEBASTIAN R., Safe reinforcement learning through meta-learned instincts, Proceedings of the ALIFE 2020: the 2020 Conference on Artificial Life, pp. 283-291, (2020)
  • [14] SUTTON R, BARTO A., Reinforcement learning: an introduction, (2017)
  • [15] SUGIYAMA M., Statistical reinforcement learning. Modern machine learning approaches, 11, 4, pp. 1330-1340, (2015)
  • [16] WANG W, YU N, GAO Y, Et al., Safe off-policy deep rein-forcement learning algorithm for volt-var control in power distribution systems, IEEE Transactions on Smart Grid, 11, 4, pp. 3008-3018, (2020)
  • [17] BAIRD L., Residual algorithms: reinforcement learning with function approximation, (1995)
  • [18] LI Bin, PENG Shurong, PENG Junzhe, Et al., Probability density prediction of wind power based on deep learning quantile regression model, Electric Power Automation Equipment, 38, 9, pp. 15-20, (2018)
  • [19] CHEN Xiliang, CAO Lei, LI Chenxi, Et al., A deep reinforcement learning method based on re-sampling optimal cache experience replay mechanism, Control and Decision, 33, 4, pp. 600-606, (2018)
12