Parameter convergence for EM and MM algorithms

It is well known that the likelihood sequence of the EM algorithm is non-decreasing and convergent (Dempster, Laird and Rubin (1977)), and that the limit points of the EM algorithm are stationary points of the likelihood (Wu (1982)): but the issue of the convergence of the EM sequence itself has not been completely settled. In this paper we close this gap and show that under general, simple, verifiable conditions, any EM sequence is convergent. In pathological cases we show that the sequence is cycling in the limit among a finite number of stationary points with equal likelihood. The results apply equally to the optimization transfer class of algorithms (MM algorithm) of Lange, Hunter, and Yang (2000). Two different EM algorithms constructed on the same dataset illustrate the convergence and the cyclic behavior.