Binary Sparse Coding of Convolutive Mixtures for Sound Localization and Separation via Spatialization

We propose a sparse coding approach to address the problem of source-sensor localization and speech reconstruction. This approach relies on designing a dictionary of spatialized signals by projecting the microphone array recordings into the array manifolds characterized for different locations in a reverberant enclosure using the image model. Sparse representation over this dictionary enables identifying the subspace of the actual recordings and its correspondence to the source and sensor locations. The speech signal is reconstructed by inverse filtering the acoustic channels associated to the array manifolds. We provide rigorous analysis on the optimality of speech reconstruction by elucidating the links between inverse filtering and source separation followed by deconvolution. This procedure is evaluated for localization, reconstruction and recognition of simultaneous speech sources using real data recordings. The results demonstrate the effectiveness of the proposed approach and compare favorably against beamforming and independent component analysis techniques.

Computational Methods for Underdetermined Convolutive Speech Localization and Separation via Model-based Sparse Component Analysis

Afsaneh Asaei, Hervé Bourlard, Volkan Cevher, Mohammadjavad Taghizadeh

In this paper, the problem of speech source localization and separation from recordings of convolutive underdetermined mixtures is studied. The problem is cast as recovering the spatio-spectral speech information embedded in a microphone array compressed measurements of the acoustic field. A model-based sparse component analysis framework is formulated for sparse reconstruction of the speech spectra in a reverberant acoustic resulting in joint localization and separation of the individual sources. We compare and contrast the computational approaches to model-based sparse recovery exploiting spatial sparsity as well as spectral structures underlying spectrographic representation of speech signals. In this context, we explore identification of the sparsity structures at the auditory and acoustic representation spaces. The auditory structures are formulated upon the principles of structural grouping based on proximity, autoregressive correlation and harmonicity of the spectral coefficients and they are incorporated for sparse reconstruction. The acoustic structures are formulated upon the image model of multipath propagation and they are exploited to characterize the compressive measurement matrix associated with microphone array recordings. Three approaches to sparse recovery relying on combinatorial optimization, convex relaxation and Bayesian methods are studied and evaluated based on thorough experiments. The sparse Bayesian learning method is shown to yield better perceptual quality while the interference suppression is also achieved using the combinatorial approach with the advantage of offering the most efficient computational cost. Furthermore, it is demonstrated that an average autoregressive model can be learned for speech localization and exploiting the proximity structure in the form of block sparse coefficients enables accurate localization. Throughout the extensive empirical evaluation, we confirm that a large and random placement of the microphones enables significant improvement in source localization and separation performance.

Elsevier Science Bv2016

Development and assessment of a localization algorithm implemented in binaural hearing aids

Gilles André Courtois, Hervé Lissek, Patrick Marmaroli

Digital wireless microphone systems have been developed to enhance speech intelligibility for hearing aid users. The voice of the speaker of interest is picked-up close to the mouth by a body-worn microphone, and rendered in a diotic way (same signal at the two ears) in both hearing aids. This means that binaural spatial cues are missing, whereas they are known to be useful for speech perception, speaker localization and sense of immersion. Some spatial information could be included in the microphone signal if the location of the talker relative to the listener would be known. This paper reports an algorithm that performs the real-time localization and tracking of the speaker’s position, taking into account the technical constraints related to hearing aids. It has been assessed with speech signals in a conventional classroom, and has reached convincing performance.

IEEE2015

Model-based Sparse Component Analysis for Reverberant Speech Localization

Afsaneh Asaei, Hervé Bourlard, Volkan Cevher, Mohammadjavad Taghizadeh

This paper proposes a speech localization framework based on model-based sparse recovery. We compare and contrast the computational sparse optimization methods incorporating harmonicity and block structures as well as autoregressive dependencies underlying spectrographic representation of speech signals. Extensive evaluations are conducted to quantify the performance bound for localization of multiple sources from underdetermined mixtures in a reverberant environment. The results demonstrate the effectiveness of sparse Bayesian learning framework for speech source localization. Furthermore, the importance of construction layout of microphone array is investigated. The outcome of this study encourages the use of ad-hoc microphones for the data acquisition set-up.

IEEE2014

Computational Methods for Underdetermined Convolutive Speech Localization and Separation via Model-based Sparse Component Analysis

Afsaneh Asaei, Hervé Bourlard, Volkan Cevher, Mohammadjavad Taghizadeh

Elsevier Science Bv2016