Publications by Mihai Gurban | EPFL Graph Search

On dynamic stream weighting for Audio-Visual Speech Recognition

Jean-Philippe Thiran, Mihai Gurban, Virginia Estellers Casas

The integration of audio and visual information improves speech recognition performance, specially in the presence of noise. In these circumstances it is necessary to introduce audio and visual weights to control the contribution of each modality to the re ...

2012

Multimodal feature extraction and fusion for audio-visual speech recognition

Mihai Gurban

Multimodal signal processing analyzes a physical phenomenon through several types of measures, or modalities. This leads to the extraction of higher-quality and more reliable information than that obtained from single-modality signals. The advantage is two ...

EPFL2009

Information theoretic feature extraction for audio-visual speech recognition

Jean-Philippe Thiran, Mihai Gurban

The problem of feature selection has been thoroughly analyzed in the context of pattern classification, with the purpose of avoiding the curse of dimensionality. However, in the context of multimodal signal processing, this problem has been studied less. O ...

2009

Selecting relevant visual features for speechreading

Jean-Philippe Thiran, Mihai Gurban, Virginia Estellers Casas

A quantitative measure of relevance is proposed for the task of constructing visual feature sets which are at the same time relevant and compact. A feature's relevance is given by the amount of information that it contains about the problem, while compactn ...

2009

Multimodal Speaker Localization from Omnidirectional Videos

Jean-Philippe Thiran, Mihai Gurban, Pascal Reuse

The use of omnidirectional cameras for videoconferencing promises to simplify the hardware setup necessary for large groups of participants. We investigate the use of a multimodal speaker detection algorithm on audio-visual sequences captured with such a c ...

2009

Using entropy as a stream reliability estimate for audio-visual speech recognition

Jean-Philippe Thiran, Mihai Gurban

We present a method for dynamically integrating audio-visual information for speech recognition, based on the estimated reliability of the audio and visual streams. Our method uses an information theoretic measure, the entropy derived from the state probab ...

2008

Dynamic modality weighting for multi-stream HMMs in Audio- Visual Speech Recognition

Jean-Philippe Thiran, Mihai Gurban, Thomas Drugman

Merging decisions from different modalities is a crucial problem in Audio-Visual Speech Recognition. To solve this, state synchronous multi-stream HMMs have been proposed for their important advantage of incorporating stream reliability in their fusion sch ...

ACM2008

Low-Dimensional Motion Features for Audio-Visual Speech Recognition

Jean-Philippe Thiran, Mihai Gurban, Andrés Vallés

Audio-visual speech recognition promises to improve the performance of speech recognizers, especially when the audio is corrupted, by adding information from the visual modality, more specifically, from the video of the speaker. However, the number of visu ...

2007

Relevant Feature Selection for Audio-Visual Speech Recognition

Jean-Philippe Thiran, Mihai Gurban, Thomas Drugman

We present a feature selection method based on information theoretic measures, targeted at multimodal signal processing, showing how we can quantitatively assess the relevance of features from different modalities. We are able to find the features with the ...

2007

Définition et sélection d’attributs visuels pour la reconnaissance audio-visuelle de la parole

Jean-Philippe Thiran, Mihai Gurban, Andrés Vallés, Thomas Drugman

2007