Publications associées à Interaction multimodale

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Pascal Frossard, Chenglin Li, Li Wei, Qin Yang, Yuelei Li

Viewers of 360-degree videos are provided with both visual modality to characterize their surrounding views and audio modality to indicate the sound direction. Though both modalities are important for saliency prediction, little work has been done by joint ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Lakmal Buddika Meegahapola

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Keep Sensors in Check: Disentangling Country-Level Generalization Issues in Mobile Sensor-Based Models with Diversity Scores

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Machine learning models trained with passive sensor data from mobile devices can be used to perform various inferences pertaining to activity recognition, context awareness, and health and well-being. Prior work has improved inference performance through t ...

Assoc Computing Machinery2023

A Conceptual Framework for Integrating Conversational Agents in Digital Education

Juan Carlos Farah

The presence of conversational agents (or chatbots) in educational contexts has been steadily increasing over the past few years. Recent surveys have shown widespread interest in the use of chatbots in education, both for research and practice. Although th ...

EPFL2023

Haptics based collaborative steering control in automated driving environments

Tomohiro Nakade

A vehicle's steering is a particular system in that it is exposed to individual subjective reviews based on criteria that are hard to assess quantitatively. Haptic design of such systems is a prime concern that has been at the center of industrial developm ...

EPFL2023

Towards Novel Evaluation Methods for Social Dialog Systems

Ekaterina Svikhnushina

Language has shaped human evolution and led to the desire to endow machines with language abilities. Recent advancements in natural language processing enable us to achieve this breakthrough in human-machine interaction. However, introducing conversational ...

EPFL2023

A Multimodal Dataset for Automatic Edge-AI Cough Detection

David Atienza Alonso, Tomas Teijeiro Campo, Lara Orlandic, Jérôme Paul Rémy Thevenot

Counting the number of times a patient coughs per day is an essential biomarker in determining treatment efficacy for novel antitussive therapies and personalizing patient care. There is a need for wearable devices that employ multimodal sensors to perform ...

Zenodo2023

Focus on People: Five Qestions from Human-Centered Computing

Daniel Gatica-Perez

A substantial body of research in multimodal interaction has studied how people naturally interact -face-to-face and through machinesand developed technology to analyze, support, and extend such forms of interaction. The talk will share personal experience ...

Assoc Computing Machinery2022

Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Xingchen Yang

Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneo ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Barbara Bruno, Jauwairia Nasir

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of prod ...

2022