Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning

Carbon trading has emerged as an effective way to promote the renewable generation and sustainable energy development. Since carbon emissions are closely coupled to energy system, it is a challenge to design a market mechanism for joint energy and carbon trading to achieve better strategies. In this study, the energy management problem with a specific focus on joint trading in multi-microgrid system is investigated by utilizing a multi-agent deep reinforcement learning approach. Initially, a joint energy and carbon trading market is established and the dispatch optimization problem is formulated as a Markov decision process without modeling uncertainties accurately. This mechanism enables direct one-to-one energy transactions among all areas, avoiding the market clearing in traditional multi-party local energy trading markets. To enhance the learning efficiency and maintain agent privacy, an enhanced multi-agent proximal policy optimization (MAPPO) algorithm that incorporates a parameter sharing mechanism is introduced. Moreover, the recurrent neural networks (RNN) structure is leveraged to perform feature encoding for individual agents, which improves the overall feature extraction capability. Through comprehensive experiments involving various algorithms, the proposed approach reduce operating costs 14.86 \% and carbon emissions 19.04 $\%$ compared with traditional MAPPO, which validates the effectiveness and performance benefits.