Special Agents Policy Gradient In Value Decomposition-based Approach

In many real-world environments, such as soldiers and general in a battlefield, or teammates and goalkeeper in a soccer field, the "general" has a significantly stronger role than the "soldier", so that it is logical to assign higher "intelligence" and "flexibility" to the "general", we define it as special agent. Here, we propose a multi-agent reinforcement learning algorithm that provides stronger intelligence to special agent in a fully cooperative heterogeneous multi-agent environment. Similar to QMIX, we design a common monotonicity critic for all agents, but a separate actor network to improve its "intelligence" for the special agent. In this way we can improve the group's ability to cooperate by giving special agent greater ability, while ensuring that the group remains cooperative. We evaluate the above algorithm on two sets of StarCraft 2 micromanagement tasks, and the experimental results show that the algorithm has a significant advantage over baseline algorithms for tasks with significant heterogeneity.