Combining Behavior Trees with MAXQ Learning to Facilitate CGFs Behavior Modeling

In simulation based military training, behaviors of Computer generated forces (CGFs) are usually constrained by doctrine and goal hierarchy. Behavior Tree (BT) is a popular AI scripting technique to encode such behavior, but suffers from time-consuming, repetitive endeavor and lack of nuanced variations. This paper proposes a novel approach MAXQ-BT, which combines BT with MAXQ learning to facilitate constrained and adaptive behavior generation. We first allow subject matter expert (SME) to encode goal hierarchy and temporal constrains with an initial BT, then a modified MAXQ learner is combined to generate specific behavior policy for selector node subtree. Finally, the learned policy is reorganized transparently as condition nodes of original selected behavior. Preliminary experiments in a predator-prey simulation scenario show that MAXQ-BT can facilitate behavior trees generation easily for CGF to achieve better behavior performance than handcrafted products.