Equilibrium payoffs of dynamic games

We give a characterization of the equilibrium payoffs of a dynamic game, which is a stochastic game where the transition function is either one or zero and players can only use pure actions in each stage. The characterization is in terms of convex combinations of connected stationary strategies; since stationary strategies are not always connected, the equilibrium set may not be convex. We show that subgame perfection may reduce the equilibrium set.