Optimal Management of Office Energy Consumption via Q-learning Algorithm

In this paper, a Q-learning based algorithm is developed to optimize energy consumption in an office, where solar energy is introduced as the renewable source and a battery is installed as the control unit. The energy consumption in the office, regarded as the energy demand, is divided into those from sockets, lights and air-conditioners. First, the time series of realtime electricity rate, renewable energy, and energy demand are modeled by echo state networks as periodic functions. Second, given these periodic functions, a Q-learning based algorithm is developed for optimal control of the battery in the office, so that the total cost on energy from the grid is reduced. Finally, numerical analysis is conducted to show the performance of the developed algorithm.