Event-related potential

Related publications (79)

Error-related EEG potentials in brain-computer interfaces

Pierre Ferrez

People with severe motor disabilities (spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS), etc.) but with intact brain functions are somehow prisoners of their own body. They need alternative ways of communication and control to interact with their environment in their everyday life. These new tools are supposed to increase their quality of life by giving these people the opportunity to recover part of their independence. Therefore, these alternative ways have to be reliable and ergonomic to be successfully used by disabled people. Over the past two decades, numerous studies proposed electroencephalogram (EEG) activity for direct brain-computer interaction. EEG-based brain-computer interfaces (BCIs) provide disabled people with new tools for control and communication and are promising alternatives to invasive methods. However, as any other interaction modality based on physiological signals and body channels (muscular activity, speech and gestures, etc.), BCIs are prone to errors in the recognition of subject's intent, and those errors can be frequent. Indeed, even well-trained subjects rarely reach 100% of success. In contrast to other interaction modalities, a unique feature of the brain channel is that it conveys both information from which we can derive mental control commands to operate a brain-actuated device as well as information about cognitive states that are crucial for a purposeful interaction, all this on the millisecond range. One of these states is the awareness of erroneous responses, which a number of groups have recently proposed as a way to improve the performance of BCIs. However, most of these studies propose the use of error-related potentials (ErrP) following an error made by the subject himself. This thesis first describes a new kind of ErrP, the so-called interaction ErrP, that are present in the ongoing EEG following an error of the interface and no longer errors of the subject himself. More importantly, these ErrP are satisfactorily detected no more in grand averages but at the level of single trials. Indeed, the classification rates of both error and correct single trials based on error-potentials detection are on average 80%. At this level it becomes possible to introduce a kind of automatic verification procedure in the BCI: after translating the subject's intention into a control command, the BCI provides a feedback of that command, but will not transfer it to the device if ErrP follow the feedback. Experimental results presented in this thesis confirm that this new protocol greatly increases the reliability of the BCI. Furthermore, this tool turns out to be of great benefit especially for beginners who normally reach moderate performances. Indeed, filtering out wrong responses increases the user's confidence in the interface and thus accelerates mastering the control of the brain-actuated device. The second issue explored in this thesis is the practical integration of ErrP detection in a BCI. Indeed, providing a first feedback of the subject's intent, as recognized by the BCI, before eventually sending the command to the controlled device, induces additional information to process by the subject and may considerably slow down the interaction since the introduction of an automatic response rejection strongly interferes with the BCI. However, this study shows the feasibility of simultaneously and satisfactorily detecting erroneous responses of the interface and classifying motor imagination for device control at the level of single trials. The integration of an automatic error detection procedure leads to great improvements of the BCI performance. Another aspect of this thesis is to investigate the potential benefit of using neurocognitive knowledge to increase the classification rate of ErrP, and more generally the performance of the BCI. Recent findings have uncovered that ErrP are most probably generated in a deep fronto-central brain area called anterior cingulate cortex (ACC). This hypothesis is verified using a well-known inverse model called sLORETA. Indeed, the localization provided for ErrP shows clear foci of activity both in the ACC and the pre-supplementary motor area (pre-SMA). The localization results using the cortical current density (CCD) model are very similar and more importantly, this model outperforms EEG for ErrP classification. Thanks to its stability, this model is likely to be successfully used in a BCI framework. The ELECTRA model for estimating local field potentials is also tested, but classification and localization results using this method are not so encouraging. More generally, the work described here suggests that it could be possible to recognize in real time high-level cognitive and emotional states from EEG (as opposed, and in addition, to motor commands) such as alarm, fatigue, frustration, confusion, or attention that are crucial for an effective and purposeful interaction. Indeed, the rapid recognition of these states will lead to truly adaptive interfaces that customize dynamically in response to changes of the cognitive and emotional/affective states of the user.

EPFL2007

Time course of prediction errors in sequential decision making

Michael Herzog, He Xu

In reinforcement learning, an agent makes sequential decisions to maximize reward. During learning, the actual and expected outcome are compared to tell whether a decision was good or bad. The difference between the actual outcome and expected outcome is the prediction error. The prediction error can be categorised into state prediction errors (SPE) and reward prediction errors (RPE). which can serve as a teaching signal in reinforcement learning. fMRI studies revealed the brain areas where the reward prediction error and the state prediction error are computed (Haruno & Kawato 2006; McClure et al. 2003; O’Doherty et al. 2003; D’Ardenne et al. 2008; Glascher et al. 2010). Here, by using 128-channel EEG, we show when the SPE and RPE are computed. In our study, participants saw an image on the computer screen and were asked to click one out of three or four buttons, which, depending on the choice, led to the presentation of a new image until a goal image was reached. After participants have learned the path to the goal, we swapped two images. The swapped images created a SPE, which was correlated with a significant change in the frontal N1 component in the Event-Related Potential. To estimate the RPE, we fit participants’ performance to a reinforcement learning algorithm SARSA-Lambda. A time window at 200-400ms in the ERP reflected well the magnitudes of the RPEs of this algorithm (r = 0.51, p = 0.02). Our results show that the frontal P3 component in ERP reflects the reward prediction process, while the state prediction process is reflected by the frontal N1 component, which is in line with the mismatch negativity studies(Campbell et al. 2007).

2018

Neural Correlates of Sequential Decision Making

Michael Herzog, He Xu

In chess, a series of moves is made until a delayed sparse feedback (win, loss) is issued, which makes it impossible to evaluate the value of a single move. There are powerful reinforcement learning (RL) algorithms, which can cope with these sequential decision making situations. A crucial component in these algorithms is the reward prediction error (RPE), which measures the difference between the actual reward and the predicted reward. Here, we show that the RPE is well reflected in a frontal negativity of the EEG. Participants saw an image on the computer screen and were asked to click one out of, for example, four buttons, which, depending on the choice, led to the presentation of a new image until a goal state was reached. 128-channel EEG was recorded. To estimate the RPE, we fit participants' performance to state of the art RL algorithms. We chose the best fitting algorithm to estimate the RPE. Two time windows (170-286ms and 400-580ms) in the event-related potential (ERP) reflected well the magnitudes of the RPEs of this algorithm. The late time window ERP magnitude was highly correlated with the RPEs (r2=0.14, p

2017

In chess, a series of moves is made until a delayed sparse feedback (win, loss) is issued, which makes it impossible to evaluate the value of a single move. There are powerful reinforcement learning (RL) algorithms, which can cope with these sequential decision making situations. A crucial component in these algorithms is the reward prediction error (RPE), which measures the difference between the actual reward and the predicted reward. Here, we show that the RPE is well reflected in a frontal negativity of the EEG. Participants saw an image on the computer screen and were asked to click one out of, for example, four buttons, which, depending on the choice, led to the presentation of a new image until a goal state was reached. 128-channel EEG was recorded. To estimate the RPE, we fit participants’ performance to state of the art RL algorithms. We chose the best fitting algorithm. Two time windows (170-286ms and 400-580ms) in the event-related potential (ERP) reflected well the magnitudes of the RPEs of this algorithm. The late time window ERP magnitude was highly correlated with the RPEs (r2=0.14, p

2017

Error-related EEG potentials in brain-computer interfaces

Pierre Ferrez

People with severe motor disabilities (spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS), etc.) but with intact brain functions are somehow prisoners of their own body. They need alternative ways of communication and control to interact with their environment in their everyday life. These new tools are supposed to increase their quality of life by giving these people the opportunity to recover part of their independence. Therefore, these alternative ways have to be reliable and ergonomic to be successfully used by disabled people. Over the past two decades, numerous studies proposed electroencephalogram (EEG) activity for direct brain-computer interaction. EEG-based brain-computer interfaces (BCIs) provide disabled people with new tools for control and communication and are promising alternatives to invasive methods. However, as any other interaction modality based on physiological signals and body channels (muscular activity, speech and gestures, etc.), BCIs are prone to errors in the recognition of subjects intent, and those errors can be frequent. Indeed, even well-trained subjects rarely reach 100 percent of success. In contrast to other interaction modalities, a unique feature of the brain channel is that it conveys both information from which we can derive mental control commands to operate a brain-actuated device as well as information about cognitive states that are crucial for a purposeful interaction, all this on the millisecond range. One of these states is the awareness of erroneous responses, which a number of groups have recently proposed as a way to improve the performance of BCIs. However, most of these studies propose the use of error-related potentials (ErrP) following an error made by the subject himself. This thesis first describes a new kind of ErrP, the so-called interaction ErrP, that are present in the ongoing EEG following an error of the interface and no longer errors of the subject himself. More importantly, these ErrP are satisfactorily detected no more in grand averages but at the level of single trials. Indeed, the classification rates of both error and correct single trials based on error-potentials detection are on average 80 percent. At this level it becomes possible to introduce a kind of automatic verification procedure in the BCI: after translating the subjects intention into a control command, the BCI provides a feedback of that command, but will not transfer it to the device if ErrP follow the feedback. Experimental results presented in this thesis confirm that this new protocol greatly increases the reliability of the BCI. Furthermore, this tool turns out to be of great benefit especially for beginners who normally reach moderate performances. Indeed, filtering out wrong responses increases the users confidence in the interface and thus accelerates mastering the control of the brainactuated device. The second issue explored in this thesis is the practical integration of ErrP detection in a BCI. Indeed, providing a first feedback of the subjects intent, as recognized by the BCI, before eventually sending the command to the controlled device, induces additional information to process by the subject and may considerably slow down the interaction since the introduction of an automatic response rejection strongly interferes with the BCI. However, this study shows the feasibility of simultaneously and satisfactorily detecting erroneous responses of the interface and classifying motor imagination for device control at the level of single trials. The integration of an automatic error detection procedure leads to great improvements of the BCI performance. Another aspect of this thesis is to investigate the potential benefit of using neurocognitive knowledge to increase the classification rate of ErrP, and more generally the performance of the BCI. Recent findings have uncovered that ErrP are most probably generated in a deep fronto-central brain area called anterior cingulate cortex (ACC). This hypothesis is verified using a well-known inverse model called sLORETA. Indeed, the localization provided for ErrP shows clear foci of activity both in the ACC and the pre-supplementary motor area (pre-SMA). The localization results using the cortical current density (CCD) model are very similar and more importantly, this model outperforms EEG for ErrP classification. Thanks to its stability, this model is likely to be successfully used in a BCI framework. The ELECTRA model for estimating local field potentials is also tested, but classification and localization results using this method are not so encouraging. More generally, the work described here suggests that it could be possible to recognize in real time high-level cognitive and emotional states from EEG (as opposed, and in addition, to motor commands) such as alarm, fatigue, frustration, confusion, or attention that are crucial for an effective and purposeful interaction. Indeed, the rapid recognition of these states will lead to truly adaptive interfaces that customize dynamically in response to changes of the cognitive and emotional/affective states of the user.

École Polytechnique Fédérale de Lausanne2007

Spatial filters yield stable features for error-related potentials across conditions

Ricardo Andres Chavarriaga Lozano, Inaki Asier Iturrate Gil, Fumiaki Iwane

Error-related potentials (ErrP) have been increasingly studied in psychophysical experiments as well as for brain-machine interfacing. In the latter case, the generalisation capabilities of ErrP decoders is a crucial element to avoid frequent recalibration processes, thus increasing their usability. Previous studies have suggested that ErrP signals are rather stable across recording sessions. Also, studies using protocols of serial stimuli presentation show that these potentials do not change significantly with the presentation rate. Here we complement these studies by analysing the decoding generalisation capabilities. Using data from monitoring experiments, we evaluate how much the performance degrades when tested in a condition different than the one the decoder was trained with. Moreover, we compare different spatial filtering techniques to see which preprocessing steps yield less-sensitive features for ErrP decoding.

2016

Possible Effect of the Trigeminal Nerve Stimulation on Auditory Event-Related Potentials

Fiorenzo Artoni, Silvestro Micera, Maria Paola Tramonti Fantozzi

2018

Act Quickly, Decide Later: Long-latency Visual Processing Underlies Perceptual Decisions but Not Reflexive Behavior

Humans largely guide their behavior by their visual representation of the world. Recent studies have shown that visual information can trigger behavior within 150 msec, suggesting that visually guided responses to external events, in fact, precede conscious awareness of those events. However, is such a view correct? By using a texture discrimination task, we show that the brain relies on long-latency visual processing in order to guide perceptual decisions. Decreasing stimulus saliency leads to selective changes in long-latency visually evoked potential components reflecting scene segmentation. These latency changes are accompanied by almost equal changes in simple RTs and points of subjective simultaneity. Furthermore, we find a strong correlation between individual RTs and the latencies of scene segmentation related components in the visually evoked potentials, showing that the processes underlying these late brain potentials are critical in triggering a response. However, using the same texture stimuli in an antisaccade task, we found that reflexive, but erroneous, prosaccades, but not antisaccades, can be triggered by earlier visual processes. In other words: The brain can act quickly, but decides late. Differences between our study and earlier findings suggesting that action precedes conscious awareness can be explained by assuming that task demands determine whether a fast and unconscious, or a slower and conscious, representation is used to initiate a visually guided response.

2011

Exploiting task constraints for self-calibrated brain-machine interface control using error-related potentials

Inaki Asier Iturrate Gil

This paper presents a new approach for self-calibration BCI for reaching tasks using error-related potentials. The proposed method exploits task constraints to simultaneously calibrate the decoder and control the device, by using a robust likelihood function and an ad-hoc planner to cope with the large uncertainty resulting from the unknown task and decoder. The method has been evaluated in closed-loop online experiments with 8 users using a previously proposed BCI protocol for reaching tasks over a grid. The results show that it is possible to have a usable BCI control from the beginning of the experiment without any prior calibration. Furthermore, comparisons with simulations and previous results obtained using standard calibration hint that both the quality of recorded signals and the performance of the system were comparable to those obtained with a standard calibration approach.

Public Library of Science2015

Spatial covariance improves BCI performance for late ERPs components with high temporal variability

Ruslan Aydarkhanov, Ricardo Andres Chavarriaga Lozano, Lucian Andrei Gheorghe, Marija Uscumlic

Objective. Event Related Potentials (ERPs) reflecting cognitive response to external stimuli, are widely used in brain computer interfaces. ERP waveforms are characterized by a series of components of particular latency and amplitude. The classical ERP decoding methods exploit this waveform characteristic and thus achieve a high performance only if there is sufficient time- and phase-locking across trials. The required condition is not fulfilled if the experimental tasks are challenging or if it is needed to generalize across various experimental conditions. Features based on spatial covariances across channels can potentially overcome the latency jitter and delays since they aggregate the information across time. Approach. We compared the performance stability of waveform and covariance-based features as well as their combination in two simulated scenarios: 1) generalization across experiments on Error-related Potentials and 2) dealing with larger latency jitter across trials. Main results. The features based on spatial covariances provide a stable performance with a minor decline under jitter levels of up to ± 300 ms, whereas the decoding performance with waveform features quickly drops from 0.85 to 0.55 AUC. The generalization across ErrP experiments also resulted in a significantly more stable performance with covariance-based features. Significance. The results confirmed our hypothesis that covariance-based features can be used to: 1) classify more reliably ERPs with higher intrinsic variability in more challenging real-life applications and 2) generalize across related experimental protocols.

2020