Theoretical Analysis of LMS Algorithm for Time-Varying Unknown System

We analyze the behaviors of the LMS algorithm for a time-varying unknown system using a statistical-mechanical method. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under conditions in which the tapped-delay line is sufficiently long. We show the existence of an optimal step size owing to the trade-off between the noise misadjustment and the lag misadjustment. Furthermore, we obtain the exact optimal step size in the case of a white reference signal. The derived theory includes the behaviors for a time-constant unknown system as a special case.