Immersion (virtual reality) | EPFL Graph Search

Immersion into virtual reality (VR) is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment. The name is a metaphoric use of the experience of submersion applied to representation, fiction or simulation. Immersion can also be defined as the state of consciousness where a "visitor" (Maurice Benayoun) or "immersant" (Char Davies)'s awareness of physical self is transformed by being surrounded in an artificial environment; used for describing partial or complete suspension of disbelief, enabling action or reaction to stimulations encountered in a virtual or artistic environment. The greater the suspension of disbelief, the greater the degree of presence achieved. According to Ernest W. Adams, immersion can be separated into three main categories: Tactical immersion: Tactical immersion is experienced when performing tactile operations that involve skill. Players feel "in the zone" while perfecting actions that result in success. Strategic immersion: Strategic immersion is more cerebral, and is associated with mental challenge. Chess players experience strategic immersion when choosing a correct solution among a broad array of possibilities. Narrative immersion: Narrative immersion occurs when players become invested in a story, and is similar to what is experienced while reading a book or watching a movie. Staffan Björk and Jussi Holopainen, in Patterns In Game Design, divide immersion into similar categories, but call them sensory-motoric immersion, cognitive immersion and emotional immersion, respectively. In addition to these, they add a new category: spatial immersion, which occurs when a player feels the simulated world is perceptually convincing. The player feels that he or she is really "there" and that a simulated world looks and feels "real". Presence, a term derived from the shortening of the original "telepresence", is a phenomenon enabling people to interact with and feel connected to the world outside their physical bodies via technology.

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

Virtual reality (VR) is immersive not only because of visual integration, but because we can act and perform in a virtual environment (VE). Beyond the mere fact that a VR system provides images computed directly from the users' viewpoint in stereoscopy, it can also provide a natural way to interact within the VE using the entire body. Full-body motion capture allows the user's movement to be mapped to a virtual body. Such a mapping may let users feel in control of (sense of agency), own (sense of body ownership) and locate themselves inside this virtual body (sense of self-location), thus leading them to feel embodied in this virtual body whenever these conditions are met (sense of embodiment (SoE)). Hence the integration of a full body avatar in VR is essential, as it brings even more; experiencing being someone else, train to recover mobility or having superhuman abilities. Most importantly, the avatar mediates the virtual nature of the VE to make it seamless for the user. Thus, this thesis focuses on helping the users execute complex movements in VR by applying a distortion to their movements. However, such a distortion is efficient only if users, when assisted, do not experience a break in embodiment (BiE), i.e., do not abruptly lose their sense of embodiment. Moreover, the way the virtual body is perceived is not consistent between users and some users might accept or reject a virtual body more easily than others. Therefore, this thesis explores how to prevent a BiE while distorting users' movement and adapting the VR experience to each individual. To this end, we designed a system combining brain-computer interface and machine learning to detect when users experience a BiE, compute users' preferences, and adapt the VR application's parameters. We first identified the user`s threshold for perceiving a movement distortion in some critical contexts, such as articular limits. We designed a new distortion to help users execute a complex movement. We introduced a method based on machine learning to find users' preferences when using our new distortion. We also established a link between the brain's error monitoring mechanism and the cognitive process of embodiment. This link exposed a new implicit EEG marker indicating when users experience a BiE. Finally, we designed a system to calibrate the distortion implicitly based on users' preferences with all these elements. This system first consists of detecting a BiE during a continuous movement thanks to state-of-the-art brain-computer interface algorithms. Our machine learning method then computes users' preferences, and the distortion is adapted dynamically. This original approach demonstrates the use of an implicit feedback loop to help users while preventing any BiE. Our framework opens an exciting perspective for personalized and self-adapting embodiment systems.

EPFL2021

Bodily self and human grid cell-like activity: probing spatial representation with immersive virtual reality

Hyukjun Moon

Grid cells are place-modulated neurons that encode the location of an agent in space, relying on the integration of various sensorimotor signals from the body. Of note, the integration of those signals is also fundamental for the agent's conscious experience of selfhood, referred to as bodily self-consciousness (BSC). Although both grid cells and BSC systems are related to the brain mechanisms of self-location and are based on multimodal signal integration, their interplay has never been investigated. My thesis aimed at revealing their links in humans by assessing the impact of BSC modulations on grid cell-like representation (GCLR): an fMRI proxy of grid cell activity in entorhinal cortex (EC). In Study 1, I assessed whether manipulation on BSC, specifically enhancing self-identification, affects GCLR during spatial navigation in a virtual reality (VR) environment. I implemented a novel fMRI paradigm integrating a spatial navigation task with online BSC manipulation through a motion-mimicking avatar. The data showed a significant reduction of GCLR as well as improved spatial navigation performance in the condition with the self-identified avatar, linking entorhinal grid cell activity and BSC through bodily self-motion cues provided online during spatial navigation. These behavioral and neural changes were further associated with increased activity in posterior parietal and retrosplenial cortex, collectively suggesting the impact of BSC on a distributed spatial navigation networks in the human brain. Consistent with the previous BSC research, in Study1, I observed illusory drifts in experienced self-location toward the self-identified avatar. In Study2, I further investigated whether illusory self-location changes would be sufficient to elicit GCLR in the absence of active spatial navigation. Adopting the Full-body illusion (FBI) paradigm to the MRI scanner, my data extend changes in self-location related to multisensory stimulation and demonstrated that these indeed evoke GCLR. Although the amplitude of GCLR during the FBI was smaller than the one during virtual navigation, their grid orientations were similar. These data suggest that the brain mechanisms underlying the BSC-induced illusory self-location are similar and comparable, albeit weaker, to the ones during virtual navigation.Temporal Interference (TI) stimulation is a novel method that can excite or inhibit neurons in the deep brain by combining high-frequency electrical fields. Building on the previous studies, in study3, I probed the causal link between GCLR and spatial navigation by applying TI stimulation to EC of participants while they were performing a VR navigation task in the scanner. The preliminary results showed that their spatial navigation was improved by excitatory stimuli compared to the control, while it deteriorated during the inhibition. Besides, I found that modulated GCLR was also associated with improved performance in the excitatory condition. Together, these data provide the first causal link between GCLR and spatial navigation, showing potential future avenues for navigation enhancement in humans. To summarize, I investigated BSC and grid cell systems jointly in humans and, for the first time, uncovered their links through robust evidence based on both behavioral and neuroimaging data. My thesis shed new light on the research of both systems which are closely intermingled but have hitherto been considered separately.

EPFL2021

Perceptual Quality of Point Clouds with application to Compression

Evangelos Alexiou

Modern information technologies and human-centric communication systems employ advanced content representations for richer portrayals of the real world. The newly adopted imaging modalities offer additional information cues and permit the depiction of realistic sceneries, enabling immersive experiences and promoting the engagement of the user with the content. In this context, point clouds have emerged as an attractive option to represent immersive media. This type of visual data has seen a revived interest in the recent years, following the release of low-cost depth sensors and the wide integration of modern graphics processing units in mobile phones and personal computers. Point clouds can be naturally employed in extended reality applications that involve 6 degree-of-freedom interactions, allowing adjustments of the 3-D visual information in a per-point basis. At the same time, complexity reductions are promoted when compared to mesh modelling counterpart, due to the absence of connectivity information and the elimination of corresponding constraints from acquisition to rendering. Yet, the vast amount of information that is required for faithful content representations implies the necessity for efficient data structures and compression algorithms. In particular, new coding schemes must be designed in order to reduce the amount of data and by extension the costs in processing, storage, and transmission of point clouds, while lossy compression solutions should restrain degradations for more appealing results. Furthermore, adequate subjective quality evaluation methodologies tailored to the nature of this 3D imaging modality are essential in order to obtain ground-truth data, and to better understand the impact of compression and rendering artifacts in visual quality. The development of high-performing objective metrics is also fundamental to accurately predict the perceptual quality of degraded models. In this thesis, we address the aforementioned challenges by proposing new subjective quality assessment methodologies that better simulate realistic use-cases of 3D model consumption. We examine several aspects related to point cloud visualization and display means, by exploring different rendering approaches and by introducing experimental set-ups that offer different degrees of interactivity to the end-user. The behavior of human observers in 6 degrees-of-freedom virtual reality scenes is analysed, and visual attention maps are constructed using head and gaze trajectories recorded from eye-tracking experiments. Moreover, navigation data obtained from interactive subjective evaluations in desktop arrangements are exploited to improve image-based quality metrics, whose performance is examined in predicting visual impairments on point cloud contents. In the same line of research, we design novel point-based quality predictors for point cloud topology and texture degradations, and we rigorously analyse their performance using several subjectively annotated data sets. Furthermore, adopting well-established subjective evaluation methodologies, state-of-the-art compression solutions are benchmarked and best practices for rate allocation between geometry and texture encoding are derived. Lastly, a learning-based point cloud compression solution for encoding of both geometric and color information is proposed, and the impact of a series of parameters is examined on the obtained performance to pave the path for future efforts on the field.

EPFL2021

Perceptual Quality of Point Clouds with application to Compression

Evangelos Alexiou

EPFL2021

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

EPFL2021

Bodily self and human grid cell-like activity: probing spatial representation with immersive virtual reality

Hyukjun Moon

EPFL2021