SCALEX: SCALability EXploration of Multi-Agent Reinforcement Learning Agents in Grid-Interactive Efficient Buildings

Renewable energy transition and decarbonization pose significant challenges for grid-interactive efficient building communities. The optimization of intermittent renewable energy can be achieved using advanced control architecture and energy storage, enhancing energy flexibility. Reinforcement learning (RL) offers potential solutions, but its scalability and computational demands in large-scale settings remain unclear. This paper examines the scalability of Soft-Actor Critic (SAC) in multi-agent systems, comparing decentralized-independent SACs and centralized SACs using CityLearn, an OpenAI Gym environment. We consider neighborhoods consisting of 2 to 64 single-family residential buildings, each equipped with cooling and heating storage devices, domestic hot water storage devices, electrical storage devices, and solar PV systems. Our findings suggest that independent controllers outperform the centralized controller with increasing number of buildings. We also show that the performance on the building level can differ from the aggregated performance.