Concept

Multimodal interaction

Related publications (263)

A Comparative Psychophysical and EEG Study of Different Feedback Modalities for HRI

José del Rocio Millán Ruiz, Roland Siegwart, Ricardo Andres Chavarriaga Lozano, Xavier Perrin

This paper presents a comparison between six different ways to convey navigational information provided by a robot to a human. Visual, auditory, and tactile feedback modalities were selected and designed to suggest a direction of travel to a human user, wh ...

2008

Using entropy as a stream reliability estimate for audio-visual speech recognition

Jean-Philippe Thiran, Mihai Gurban

We present a method for dynamically integrating audio-visual information for speech recognition, based on the estimated reliability of the audio and visual streams. Our method uses an information theoretic measure, the entropy derived from the state probab ...

2008

Watch-dictaphone for Automatic Medical Codification

Pierre-André Farine, Sara Grassi Pauletti, Patrick Stadelmann, Frédéric David Meylan

This paper describes the development of a voice data recording watch and the accompanying software and hardware needed to build an integrated solution for medical data capture and automatic codification. The developed system is used to capture on the stand ...

EURASIP2008

Filter Bank Design based on Minimization of Individual Aliasing Terms for Minimum Mutual Information Subband Adaptive Beamforming

Philip Neil Garner, Weifeng Li

This paper presents new filter bank design methods for sub- band adaptive beamforming. In this work, we design analysis and synthesis prototypes for modulated filter banks so as to minimize each aliasing term individually. We then drive the total response ...

2008

Dynamic modality weighting for multi-stream HMMs in Audio- Visual Speech Recognition

Jean-Philippe Thiran, Mihai Gurban, Thomas Drugman

Merging decisions from different modalities is a crucial problem in Audio-Visual Speech Recognition. To solve this, state synchronous multi-stream HMMs have been proposed for their important advantage of incorporating stream reliability in their fusion sch ...

ACM2008

Silence Models in Weighted Finite-State Transducers

Philip Neil Garner

We investigate the effects of different silence modelling strategies in Weighted Finite-State Transducers for Automatic Speech Recognition. We show that the choice of silence models, and the way they are included in the transducer, can have a significant e ...

2008

Silence Models in Weighted Finite-State Transducers

Philip Neil Garner

IDIAP2008

On the modelling of multi-modal data using redundant dictionaries

Gianluca Monaci

Real-world phenomena involve complex interactions between multiple signal modalities. As a consequence, humans are used to integrate at each instant perceptions from all their senses in order to enrich their understanding of the surrounding world. This par ...

EPFL2007

Reliability-based decision fusion in multimodal biometric verification systems

Jonas Richiardi, Andrzej Drygajlo, Krzysztof Kryszczuk, Plamen Prodanov

We present a methodology of reliability estimation in the multimodal biometric verification scenario. Reliability estimation has shown to be an efficient and accurate way of predicting and correcting erroneous classification decisions in both unimodal (spe ...

2007

Learning Multi-Modal Dictionaries

Pierre Vandergheynst, Rémi Gribonval, Gianluca Monaci, Philippe Jost

2007

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.