On the learning mechanism of adaptive filters

This paper highlights, both analytically and by simulations, some interesting phenomena regarding the behavior of ensemble-average learning curves of adaptive filters that may have gone unnoticed. Among other results, the paper shows that even ensemble-average learning curves of single-tap LMS filters actually exhibit two distinct rates of convergence: one for the initial time instants and another, faster one, for later time instants. In addition, such curves tend to converge faster than predicted by mean-square theory and can converge even when a mean-square stability analysis predicts divergence. These effects tend to be magnified by increasing the step size. Two of the conclusions that follow from this work are (1) the mean-square stability alone may not be the most appropriate performance measure, especially for larger step sizes. A combination of mean-square stability and almost sure (a.s.) stability seems to be more appropriate. (2) Care is needed while interpreting ensemble-average curves for larger step sizes. The curves can lead to erroneous conclusions unless a large number of experiments are averaged (at times of the order of tens of thousands or higher).