Shared electrophysiology mechanisms of body ownership and motor imagery

Although we feel, see, and experience our hands as our own (body or hand ownership), recent research has shown that illusory hand ownership can be induced for fake or virtual hands and may useful for neuroprosthetics and brain-computer interfaces. Despite the vast amount of behavioral data on illusory hand ownership, neuroimaging studies are rare, in particular electrophysiological studies. Thus, while the neural systems underlying hand ownership are relatively well described, the spectral signatures of body ownership as measured by electroencephalography (EEG) remain elusive. Here we induce illusory hand ownership in an automated, computer-controlled manner using virtual reality while recording 64-channel EEG and found that illusory hand ownership is reflected by a body-specific modulation in the mu-band over fronto-parietal cortex. In a second experiment in the same subjects, we then show that mu as well as beta-band activity in highly similar fronto-parietal regions was also modulated during a motor imagery task often used in paradigms employing non-invasive brain-computer interface technology. These data provide insights into the electrophysiological brain mechanisms of illusory hand ownership and their strongly overlapping mechanisms with motor imagery in fronto-parietal cortex. They also highlight the potential of combining high-resolution EEG with virtual reality setups and automatized stimulation protocols for systematic, reproducible stimulus presentation in cognitive neuroscience, and may inform the design of non-invasive brain-computer interfaces.

Fully automated modulation of virtual-body ownership supresses fronto-parietal mu rhythm

Olaf Blanke, Nathaniel Evans

After continuously applying spatially and temporally congruent visuo-tactile stimulation to external, body-like objects, healthy humans feel the fake part to be the origin of the touch sensations and feel it to be their own. Incongruent stimulation nullifies both illusory changes. This Rubber Hand Illusion (RHI) is the most used paradigm to experimentally investigate healthy humans’ bodily self-consciousness and the extension of body ownership to foreign objects [1]. Standard RHI setups are limited by physical constraints and suffer the disadvantage of imprecise temporo-spatial stimulus control. Further, little is known about the neural and electrophysiological mechanisms associated with the crossmodal conflict and the induced changes in body ownership during the RHI. Here we recorded 64-channel electroencephalography (EEG) to investigate the electrophysiological changes while exposed to a carefully controlled, bilateral RHI induced by haptics and virtual reality (VR) technology. Methods Participants (n=8) sat at a table with a chin rest. Arms were placed (palms up) underneath the table to rest on the legs. A virtual scene depicting the real-world experimental scene was displayed on an immersive Head Mounted Display (HMD). We employed a 2x2 factorial design (Object, Stroking). Participants saw either two arms or two non-body cylinders (Object) projecting from their shoulders onto a virtual table, seeing the virtual arms or cylinder in a position ~20cm above their real arms. Visuo-tactile stimulation was provided by four vibration motors affixed to the palms of the (unseen) left and right hands and by animation of corresponding virtual motors on the virtual hands. Two visuo-tactile modes were defined (Stroking): 1) synchronous: virtual visual motors were shown to vibrate in temporal and spatial synchrony with the tactile vibration motors and 2) asynchronous: virtual visual motors were shown to vibrate with a temporal delay (100±50ms) and a randomly selected spatial direction with respect to the tactile vibration motor pattern. Two additional baseline conditions were recorded (no visual animation of the virtual motors and no tactile stimulation; no visual animation of the virtual motors with tactile stimulation). Subjective experience of the illusion was gauged by a questionnaire. 64-channel EEG was sampled at 2048Hz. Bipolar electrooculograms were recorded for later artifact removal. EEG analysis consisted of breaking 30s of spontaneous EEG data per condition into epochs of 2s. Epochs contaminated with eye blink artifacts or transient changes in electrode-to-scalp conductance were removed. Spectral power changes were computed in three frequency bands: alpha (8-13 Hz), beta (14-30 Hz) and gamma (30-100 Hz). Statistical analysis was performed at the scalp level and included a suprathreshold cluster permutation test to control for Type I errors. The generators of the significant scalp maps were localized with an inverse solution (sLORETA) [2]. Results Repeated ANOVA analysis of the questionnaire indicated a significant interaction between synchrony and question (F=1.65, p=0.03), and a main synchrony effect (F=40.29, p

2010

Methods for Single Trial Analysis of Asynchronous EEG Patterns

Nicolas Bourdaud

Processing of electroencephalographic (EEG) signals has mostly focused on analysing correlates that are time-locked to an observable event. However, when the signal is acquired in less controlled environment, like in the context of a brain-computer interface operating in the real-world, this synchronous nature does not hold any longer. The analysis of such signal requires the design of methods that rely less on time-locked nature. These methods are also the requirements for study endogenous processes for which the ground truth of when the process take place in time is not available. In this thesis, we present methods to analyse brain signals, EEG in particular, that are not time-locked to observable events. This thesis documents three major contributions : (i) it proposes a Bayesian formalism to the problem of asynchronous EEG pattern classification, (ii) it shows the importance of generative models to achieve this task and (iii) it shows that such methods can be used to gather information and classify the EEG correlates of decision-making process while classical methods fail at it. First, we propose methods to handle non-time-locked EEG patterns by making the hypothesis that, in each trial, only a part of the signal contains the relevant pattern of interest. This relevant part can appear at any time in the analysis window and differently for each trial. The rest of the trial corresponds to a non-informative part irrelevant to the targeted cognitive task. Starting from a discriminant asynchronous approach handling independently the time-samples in the trial, we extend this method to a generative Bayesian model where each part is formally modelled. This is a main difference compared to the classical approach which usually try to avoid to model the non-informative part. Then, making the assumption that the informative part can be modelled by a time sequence, we adapt the previous method to a Bayesian model of asynchronous template matching which allows the recognition of the time onset of the pattern of interest in each trial. Second, we show the importance of the generative model which, thanks to the Bayesian approach allows us to alleviate the problem of choosing of the hyperparameters of the initial discriminative approach. Compared to the initial discriminant model, us- ing a generative approach leads to use more parameters into the model but whose estimation is helped by the prior we provide. By doing so, we provide a more intuitive way for the experimenter to adapt the method to other problems. Using a generative model and a Bayesian estimation also enables us to improve the generalisation of the model of asynchronous template matching. This model has indeed been tested as benchmark on jittered evoked potential data and has shown to successfully improve the signal-to-noise ratio, recover the evoked response and classify better than classical methods. Finally, we see the importance of asynchronous methods for classification of the EEG correlates of decision-making process. We test this in the context of the study of the exploration/exploitation contrast. Exploration is related to decision making in an uncertain environment. This situation arises a conflict between two opposing needs : gathering information about the environment and exploiting this knowledge in order to optimise the decision. Using an experimental setup that forces the subjects to switch between exploratory or exploitative actions, we show for the first time that it is possible to classify the EEG correlates of the exploratory behaviour. Moreover, we show that synchronous methods fail at classify this contrast thus requiring advanced asynchronous ones. The results also confirm that the brain areas relevant to this switch are mainly the left parietal and medial frontal cortex which is consistent with the neurophysiological findings based on functional magnetic resonance imagery. In addition we have been able to show the importance of alpha rhythm for this contrast. In summary, this thesis provides a formal framework for classification of asynchronous EEG patterns using a generative Bayesian approach. It also provides a methodology to approach the study of EEG correlates of cognitive tasks when little is known about them and when the targeted pattern is reasonably assumed to be non time-locked.

EPFL2012

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

Virtual reality (VR) is immersive not only because of visual integration, but because we can act and perform in a virtual environment (VE). Beyond the mere fact that a VR system provides images computed directly from the users' viewpoint in stereoscopy, it can also provide a natural way to interact within the VE using the entire body. Full-body motion capture allows the user's movement to be mapped to a virtual body. Such a mapping may let users feel in control of (sense of agency), own (sense of body ownership) and locate themselves inside this virtual body (sense of self-location), thus leading them to feel embodied in this virtual body whenever these conditions are met (sense of embodiment (SoE)). Hence the integration of a full body avatar in VR is essential, as it brings even more; experiencing being someone else, train to recover mobility or having superhuman abilities. Most importantly, the avatar mediates the virtual nature of the VE to make it seamless for the user. Thus, this thesis focuses on helping the users execute complex movements in VR by applying a distortion to their movements. However, such a distortion is efficient only if users, when assisted, do not experience a break in embodiment (BiE), i.e., do not abruptly lose their sense of embodiment. Moreover, the way the virtual body is perceived is not consistent between users and some users might accept or reject a virtual body more easily than others. Therefore, this thesis explores how to prevent a BiE while distorting users' movement and adapting the VR experience to each individual. To this end, we designed a system combining brain-computer interface and machine learning to detect when users experience a BiE, compute users' preferences, and adapt the VR application's parameters. We first identified the user`s threshold for perceiving a movement distortion in some critical contexts, such as articular limits. We designed a new distortion to help users execute a complex movement. We introduced a method based on machine learning to find users' preferences when using our new distortion. We also established a link between the brain's error monitoring mechanism and the cognitive process of embodiment. This link exposed a new implicit EEG marker indicating when users experience a BiE. Finally, we designed a system to calibrate the distortion implicitly based on users' preferences with all these elements. This system first consists of detecting a BiE during a continuous movement thanks to state-of-the-art brain-computer interface algorithms. Our machine learning method then computes users' preferences, and the distortion is adapted dynamically. This original approach demonstrates the use of an implicit feedback loop to help users while preventing any BiE. Our framework opens an exciting perspective for personalized and self-adapting embodiment systems.

EPFL2021