Concept

Multimodal interaction

Related publications (263)

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Pascal Frossard, Chenglin Li, Li Wei, Qin Yang, Yuelei Li

Viewers of 360-degree videos are provided with both visual modality to characterize their surrounding views and audio modality to indicate the sound direction. Though both modalities are important for saliency prediction, little work has been done by joint ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Keep Sensors in Check: Disentangling Country-Level Generalization Issues in Mobile Sensor-Based Models with Diversity Scores

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Machine learning models trained with passive sensor data from mobile devices can be used to perform various inferences pertaining to activity recognition, context awareness, and health and well-being. Prior work has improved inference performance through t ...

New York2023

A Conceptual Framework for Integrating Conversational Agents in Digital Education

Juan Carlos Farah

The presence of conversational agents (or chatbots) in educational contexts has been steadily increasing over the past few years. Recent surveys have shown widespread interest in the use of chatbots in education, both for research and practice. Although th ...

EPFL2023

Haptics based collaborative steering control in automated driving environments

Tomohiro Nakade

A vehicle's steering is a particular system in that it is exposed to individual subjective reviews based on criteria that are hard to assess quantitatively. Haptic design of such systems is a prime concern that has been at the center of industrial developm ...

EPFL2023

A Multimodal Dataset for Automatic Edge-AI Cough Detection

David Atienza Alonso, Tomas Teijeiro Campo, Lara Orlandic, Jérôme Paul Rémy Thevenot

Counting the number of times a patient coughs per day is an essential biomarker in determining treatment efficacy for novel antitussive therapies and personalizing patient care. There is a need for wearable devices that employ multimodal sensors to perform ...

Zenodo2023

Fashioning the Future: Unlocking the Creative Potential of Deep Generative Models for Design Space Exploration

Pierre Dillenbourg, Richard Lee Davis, Kevin Gonyop Kim, Thiemo Wambsganss, Wei Jiang

This paper investigates the potential impact of deep generative models on the work of creative professionals, specifically focusing on fashion design. We argue that current generative modeling tools lack critical features that would make them useful creati ...

Association for Computing Machinery2023

The role of the visual field size in artificial vision

Diego Ghezzi

Objective. Artificial vision has been and still is the subject of intense research. The ultimate goal is to help blind people in their daily life. Approaches to artificial vision, including visual prostheses and optogenetics, have strongly focused on resto ...

IOP Publishing Ltd2023

Focus on People: Five Qestions from Human-Centered Computing

Daniel Gatica-Perez

A substantial body of research in multimodal interaction has studied how people naturally interact -face-to-face and through machinesand developed technology to analyze, support, and extend such forms of interaction. The talk will share personal experience ...

New York2022

Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Xingchen Yang

Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneo ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Barbara Bruno, Jauwairia Nasir

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of prod ...

2022

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.