Cooperating in Long-term Relationships with Time-Varying Structure

Extended interactions between agents have commonly been studied in the context of repeated games (RGs), in which the same players repeatedly interact in the same scenario. However, such interactions are uncommon in practice. Typically, the players' goals, action sets, and payoffs change from encounter to encounter, often in ways the players cannot easily model or control. These more realistic interactions, which we model as a form of stochastic game called interaction games (IGs), have attributes which prohibit the straight-forward application of many often-used algorithms developed for RGs. In this paper, we generalize several algorithms previously designed for RGs, and explore their behavior and performance in IGs. Our results suggest that at least some of the methodologies designed for RGs can, with some modifications, be extended to IGs.