Dynamic Programming or Direct Comparison?

The standard approach to stochastic control is dynamic programming. In our recent research, we proposed an alternative approach based on direct comparison of the performance of any two policies. This approach has a number of advantages: the results may be derived in a simple and intuitive way; the approach applies to different optimization problems, including finite and infinite horizon, discounting and average performance, discrete time discrete states and continuous time and continuous stats, etc., in the same way; and it may be generalized to some non-standard problems where dynamic programming fails. This approach also links stochastic control to perturbation analysis, reinforcement learning and other research subjects in optimization, which may stimulate new research directions.