Minimizing Energy Cost in PV Battery Storage Systems Using Reinforcement Learning

This article addresses the development and tuning of an energy management for a photovoltaic (PV) battery storage system for the cost-optimized use of PV energy using reinforcement learning (RL). An innovative energy management concept based on the Proximal Policy Optimization algorithm in combination with recurrent Long Short-Term Memory neural networks is developed for data-based policy learning, a concept that has been rarely addressed in the literature so far. As a reference system for the simulation-based investigations, a PV battery storage system is modelled, parametrized and implemented with an interface for the RL algorithm. To demonstrate the generalization capability of the learned energy management, 98 training and 12 evaluation episodes, each with a length of one year, are generated from an empirical dataset of global radiation and load power time series. To improve the convergence speed and stability of the RL algorithm as well as the learned policy with regards to techno-economic metrics, an extensive hyperparameter study is conducted by training 216 control policies with different hyperparameter configurations. A simulation-based evaluation of the learned energy management against conventional rule-based and model-predictive energy managements shows that the RL-based concept can achieve slightly better results in terms of energy costs and the amount of energy fed into the grid than the commonly used model-predictive method.