Experience | EPFL Graph Search

Experience refers to conscious events in general, more specifically to perceptions, or to the practical knowledge and familiarity that is produced by these processes. Understood as a conscious event in the widest sense, experience involves a subject to which various items are presented. In this sense, seeing a yellow bird on a branch presents the subject with the objects "bird" and "branch", the relation between them and the property "yellow". Unreal items may be included as well, which happens when experiencing hallucinations or dreams. When understood in a more restricted sense, only sensory consciousness counts as experience. In this sense, experience is usually identified with perception and contrasted with other types of conscious events, like thinking or imagining. In a slightly different sense, experience refers not to the conscious events themselves but to the practical knowledge and familiarity they produce. In this sense, it is important that direct perceptual contact with th

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

Virtual reality (VR) is immersive not only because of visual integration, but because we can act and perform in a virtual environment (VE). Beyond the mere fact that a VR system provides images computed directly from the users' viewpoint in stereoscopy, it can also provide a natural way to interact within the VE using the entire body. Full-body motion capture allows the user's movement to be mapped to a virtual body. Such a mapping may let users feel in control of (sense of agency), own (sense of body ownership) and locate themselves inside this virtual body (sense of self-location), thus leading them to feel embodied in this virtual body whenever these conditions are met (sense of embodiment (SoE)). Hence the integration of a full body avatar in VR is essential, as it brings even more; experiencing being someone else, train to recover mobility or having superhuman abilities. Most importantly, the avatar mediates the virtual nature of the VE to make it seamless for the user. Thus, this thesis focuses on helping the users execute complex movements in VR by applying a distortion to their movements. However, such a distortion is efficient only if users, when assisted, do not experience a break in embodiment (BiE), i.e., do not abruptly lose their sense of embodiment. Moreover, the way the virtual body is perceived is not consistent between users and some users might accept or reject a virtual body more easily than others. Therefore, this thesis explores how to prevent a BiE while distorting users' movement and adapting the VR experience to each individual. To this end, we designed a system combining brain-computer interface and machine learning to detect when users experience a BiE, compute users' preferences, and adapt the VR application's parameters. We first identified the user`s threshold for perceiving a movement distortion in some critical contexts, such as articular limits. We designed a new distortion to help users execute a complex movement. We introduced a method based on machine learning to find users' preferences when using our new distortion. We also established a link between the brain's error monitoring mechanism and the cognitive process of embodiment. This link exposed a new implicit EEG marker indicating when users experience a BiE. Finally, we designed a system to calibrate the distortion implicitly based on users' preferences with all these elements. This system first consists of detecting a BiE during a continuous movement thanks to state-of-the-art brain-computer interface algorithms. Our machine learning method then computes users' preferences, and the distortion is adapted dynamically. This original approach demonstrates the use of an implicit feedback loop to help users while preventing any BiE. Our framework opens an exciting perspective for personalized and self-adapting embodiment systems.

EPFL2021

CoPainter: Interactive AI drawing experience

Pawel Golinski

Google AI Experiments [3] is a set of simple experiments that showcase innova- tive uses of AI in UX. As part of this collection, Google shared Quick Draw, where user draws simple sketches and intelligent agent tries to guess what it depicts, and Sketch- RNN interactive demo, an experience where user starts a sketch and recurrent neural network model completes it [4]. These experiences provide fun and engaging drawing activities which have potential to be used in more educational context. This report describes CoPainter - an interactive AI drawing experience which aims to provide an educational activity based on learning by teaching paradigm - the application asks the user to draw a painting of some object in cooperation with an intel- ligent agent (which can either be a robot or an abstract entity). The activity is based on Sketch-RNN model mentioned above. Main contributions of this project are porting the Sketch-RNN web app experience to Qt, which enables integration with CoWriter code- base and makes it possible to collect more data on user activities. An additional draw in turns game mode is also introduced. The activity may possibly be extended to enable it to be used in diagnosing and treating dysgraphia, which is briefly mentioned in this re- port and is aim of further research on this project.

2019

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

EPFL2021