Stationary Equilibria in Discounted Stochastic Games

We provide a new characterisation of the set of stationary equilibria for finite discounted N-player stochastic games, based on the definition of an auxiliary one-shot game with the same set of equilibria. This result is the extension, to the N-player case, of a similar characterisation for two-player zero-sum stochastic games (Attia and Oliu-Barton in Proc Natl Acad Sci USA 116:26435–26443, 2019) which led to a tractable formula for the limit value. Though the general case presents additional challenges, our characterisation may have further applications, notably in terms of the description and computation of stationary equilibria and of their limit as the discount rates vanish.