Constructing a hierarchical ontology for reinforcement learning multi-agent system

In RL agent system, agents learn from environment, sense the dynamic environment, and get feedback in order to instruct their behaviors. The important issue for MAS is the system architecture, while the crucial problem of software architecture is how to describe the dynamic external environment for RL MAS unambiguously. Ontology is a logical theory, which includes intension and extension and gives partial and explicit conceptualization account for entity. In this article, we construct a hierarchical ontology for RL MAS from 3 aspects: architecture, application and implementation. According to the definition of software architecture, the component ontology should consist of judgment, interpret, encapsulating, graphic element and history Database agents. The connection ontology consists of link agents, and the configuration ontology is different from coarse or fine granularity of agents for electric system; Concerning the equilibrium between usability and reusability of the application ontology, there are 3 layers: abstract concept ontology layer for typical multiple-tire structure, domain ontology layer which deals with drawing graphic for generating and distributing electricity, ontology transfer layer is suitable for the dynamic changing domain. The implementation ontology includes selecting candidate terms and synonymous, and linking them together as a chain in RDF (Resource Description Framework) formally.