The timing of exploratory decision-making revealed by single-trial topographic EEG analyses

Decision-making in an uncertain environment is driven by two major needs: exploring the environment to gather information or exploiting acquired knowledge to maximize reward. The neural processes underlying exploratory decision-making have been mainly studied by means of functional magnetic resonance imaging, overlooking any information about the time when decisions are made. Here, we carried out an electroencephalography (EEG) experiment, in order to detect the time when the brain generators responsible for these decisions have been sufficiently activated to lead to the next decision. Our analyses, based on a classification scheme, extract time-unlocked voltage topographies during reward presentation and use them to predict the type of decisions made on the subsequent trial. Classification accuracy, measured as the area under the Receiver Operator's Characteristic curve was on average 0.65 across 7 subjects. Classification accuracy was above chance levels already after 516 ms on average, across subjects. We speculate that decisions were already made before this critical period, as confirmed by a positive correlation with reaction times across subjects. On an individual subject basis, distributed source estimations were performed on the extracted topographies to statistically evaluate the neural correlates of decision-making. For trials leading to exploration, there was significantly higher activity in dorsolateral prefrontal cortex and the right supramarginal gyrus; areas responsible for modulating behavior under risk and deduction. No area was more active during exploitation. We show for the first time the temporal evolution of differential patterns of brain activation in an exploratory decision-making task on a single-trial basis.

Methods for Single Trial Analysis of Asynchronous EEG Patterns

Nicolas Bourdaud

Processing of electroencephalographic (EEG) signals has mostly focused on analysing correlates that are time-locked to an observable event. However, when the signal is acquired in less controlled environment, like in the context of a brain-computer interface operating in the real-world, this synchronous nature does not hold any longer. The analysis of such signal requires the design of methods that rely less on time-locked nature. These methods are also the requirements for study endogenous processes for which the ground truth of when the process take place in time is not available. In this thesis, we present methods to analyse brain signals, EEG in particular, that are not time-locked to observable events. This thesis documents three major contributions : (i) it proposes a Bayesian formalism to the problem of asynchronous EEG pattern classification, (ii) it shows the importance of generative models to achieve this task and (iii) it shows that such methods can be used to gather information and classify the EEG correlates of decision-making process while classical methods fail at it. First, we propose methods to handle non-time-locked EEG patterns by making the hypothesis that, in each trial, only a part of the signal contains the relevant pattern of interest. This relevant part can appear at any time in the analysis window and differently for each trial. The rest of the trial corresponds to a non-informative part irrelevant to the targeted cognitive task. Starting from a discriminant asynchronous approach handling independently the time-samples in the trial, we extend this method to a generative Bayesian model where each part is formally modelled. This is a main difference compared to the classical approach which usually try to avoid to model the non-informative part. Then, making the assumption that the informative part can be modelled by a time sequence, we adapt the previous method to a Bayesian model of asynchronous template matching which allows the recognition of the time onset of the pattern of interest in each trial. Second, we show the importance of the generative model which, thanks to the Bayesian approach allows us to alleviate the problem of choosing of the hyperparameters of the initial discriminative approach. Compared to the initial discriminant model, us- ing a generative approach leads to use more parameters into the model but whose estimation is helped by the prior we provide. By doing so, we provide a more intuitive way for the experimenter to adapt the method to other problems. Using a generative model and a Bayesian estimation also enables us to improve the generalisation of the model of asynchronous template matching. This model has indeed been tested as benchmark on jittered evoked potential data and has shown to successfully improve the signal-to-noise ratio, recover the evoked response and classify better than classical methods. Finally, we see the importance of asynchronous methods for classification of the EEG correlates of decision-making process. We test this in the context of the study of the exploration/exploitation contrast. Exploration is related to decision making in an uncertain environment. This situation arises a conflict between two opposing needs : gathering information about the environment and exploiting this knowledge in order to optimise the decision. Using an experimental setup that forces the subjects to switch between exploratory or exploitative actions, we show for the first time that it is possible to classify the EEG correlates of the exploratory behaviour. Moreover, we show that synchronous methods fail at classify this contrast thus requiring advanced asynchronous ones. The results also confirm that the brain areas relevant to this switch are mainly the left parietal and medial frontal cortex which is consistent with the neurophysiological findings based on functional magnetic resonance imagery. In addition we have been able to show the importance of alpha rhythm for this contrast. In summary, this thesis provides a formal framework for classification of asynchronous EEG patterns using a generative Bayesian approach. It also provides a methodology to approach the study of EEG correlates of cognitive tasks when little is known about them and when the targeted pattern is reasonably assumed to be non time-locked.

EPFL2012

Belief as a Wise Wager

Leyla Loued-Khenissi

From the moment we wake up in the morning to the day's ebb when we settle in to sleep, we are bound to the task of decision-making. Some of these decisions barely register in our consciousness, if at all, while others, less shy, take a more prominent place at our mind's table. Regardless of the importance or difficulty of the decision, few, if any, are made with perfect information: being such a small of part of a large system, we can only know so much. Further, the system itself sends us noisy information for us to encode and decode as best we can. How do we do this? How do we continuously and, for the most part successfully, resolve uncertainty in order to survive and even flourish? We propose to define uncertainty in decision-making as a computational process, in line with information-processing theories of neural mechanisms. To that end, we investigate the neural correlates of uncertainty processing using functional magnetic resonance imaging (fMRI) in humans within a predictive coding framework. The field has already produced considerable evidence showing that decisions are made with the aim of maximizing utility, a process involving the dopaminergic reward system. We turn our focus to the uncertainty surrounding predictions and their concomitant errors by conducting a two-part fMRI experiment on 23 subjects. In the first session, we elicited objective, cognitive (financial) uncertainty in a gambling task. In the second session, we exposed individuals to subjective, perceptual uncertainty, in the form of visual illusions. Our fMRI results, modeled by computational definitions of surprise, confidence and information, show that 1) the brain employs computational principles to resolve uncertainty; 2) certain regions are consistently implicated in processing said uncertainty, notably insular cortex regions, across modalities (cognitive and perceptual), be it of a subjective or objective nature. These findings support the notion that the brain is an active inference machine, a paradigm within which further aspects of cognition can be investigated.

EPFL2018

Characterizing the EEG Correlates of Exploratory Behavior

Nicolas Bourdaud, Ricardo Andres Chavarriaga Lozano

This study aims to characterize the EEG correlates of exploratory behavior. Decision making in an uncertain environment raises a conflict between two opposing needs: gathering information about the environment and exploiting this knowledge in order to optimize the decision. Exploratory behavior has already been studied using fMRI. Based on a usual paradigm in reinforcement learning, this study has shown bilateral activation in the frontal and parietal cortex. To our knowledge, no previous study has been done on it using EEG. The study of the exploratory behavior using EEG signals raises two difficulties. First, the labels of trial as exploitation or exploration cannot be directly derived from the subject action. In order to access this information, a model of how the subject makes his decision must be built. The exploration related information can be then derived from it. Second, because of the complexity of the task, its EEG correlates are not necessarily time locked with the action. So the EEG processing methods used should be designed in order to handle signals that shift in time across trials. Using the same experimental protocol as the fMRI study, results show that the bilateral frontal and parietal areas are also the most discriminant. This strongly suggests that the EEG signal also conveys information about the exploratory behavior.