On the Optimality of Operator-Like Wavelets for Sparse AR(1) Processes

Sinusoidal transforms such as the DCT are known to be optimal—that is, asymptotically equivalent to the Karhunen-Loève transform (KLT)—for the representation of Gaussian stationary processes, including the classical AR(1) processes. While the KLT remains applicable for non-Gaussian signals, it loses optimality and, is outperformed by the independent-component analysis (ICA), which aims at producing the most-decoupled representation. In this paper, we consider an extension of the classical AR(1) model that is driven by symmetric-alpha-stable (SαS) noise which is either Gaussian (α = 2) or sparse (0 < α < 2). For the sparse (non-Gaussian) regime, we prove that an expansion in a proper wavelet basis (including the Haar transform) is much closer to the optimal orthogonal ICA solution than the classical Fourier-type representations. Our criterion for optimality, which favors independence, is the Kullback-Leibler divergence between the joint pdf of the original signal and the product of the marginals in the transformed domain. We also observe that, for very sparse AR(1) processes (α ≤ 1), the operator-like wavelet transform is indistinguishable from the ICA solution that is determined through numerical optimization.