Uncertainty-aware Model Inversion Networks

In this thesis, we assess a new framework called UMIN on a data-driven optimization problem. Such a problem happens recurrently in real life and can quickly become dicult to model when the input has a high dimensionality as images for instance. From the architecture of aircraft to the design of proteins, a great number of dierent techniques have already been explored. Based on former solutions, this work introduces a brand new Bayesian approach that updates previous frameworks. Former model architectures use generative adversarial networks on one side and a forward model on the other side to improve the accuracy of the results. However, employing a Bayesian network allows us to leverage its uncertainty estimates to enhance the accuracy of the results and also to reduce unrealistic samples output by the generator. By creating new experiments on a modern MNIST dataset and by reproducing former works taken as baseline, we show that the framework introduces in this work outperforms the previous method. The whole code is available at the following url: https://github.com/RomainGratier/Black-box_Optimization_via_Deep_ Generative-Exploratory_Networks.

Bayesian Inference for Spiking Neuron Models with a Sparsity Prior

Matthias Seeger

Generalized linear models are the most commonly used tools to describe the stimulus selectivity of sensory neurons. Here we present a Bayesian treatment of such models. Using the expectation propagation algorithm, we are able to approximate the full posterior distribution over all weights. In addition, we use a Laplacian prior to favor sparse solutions. Therefore, stimulus features that do not critically influence neural activity will be assigned zero weights and thus be effectively excluded by the model. This feature selection mechanism facilitates both the interpretation of the neuron model as well as its predictive abilities. The posterior distribution can be used to obtain confidence intervals which makes it possible to assess the statistical significance of the solution. In neural data analysis, the available amount of experimental measurements is often limited whereas the parameter space is large. In such a situation, both regularization by a sparsity prior and uncertainty estimates for the model parameters are essential. We apply our method to multi-electrode recordings of retinal ganglion cells and use our uncertainty estimate to test the statistical significance of functional couplings between neurons. Furthermore we used the sparsity of the Laplace prior to select those filters from a spike-triggered covariance analysis that are most informative about the neural response.

2008

Methods for Single Trial Analysis of Asynchronous EEG Patterns

Nicolas Bourdaud

Processing of electroencephalographic (EEG) signals has mostly focused on analysing correlates that are time-locked to an observable event. However, when the signal is acquired in less controlled environment, like in the context of a brain-computer interface operating in the real-world, this synchronous nature does not hold any longer. The analysis of such signal requires the design of methods that rely less on time-locked nature. These methods are also the requirements for study endogenous processes for which the ground truth of when the process take place in time is not available. In this thesis, we present methods to analyse brain signals, EEG in particular, that are not time-locked to observable events. This thesis documents three major contributions : (i) it proposes a Bayesian formalism to the problem of asynchronous EEG pattern classification, (ii) it shows the importance of generative models to achieve this task and (iii) it shows that such methods can be used to gather information and classify the EEG correlates of decision-making process while classical methods fail at it. First, we propose methods to handle non-time-locked EEG patterns by making the hypothesis that, in each trial, only a part of the signal contains the relevant pattern of interest. This relevant part can appear at any time in the analysis window and differently for each trial. The rest of the trial corresponds to a non-informative part irrelevant to the targeted cognitive task. Starting from a discriminant asynchronous approach handling independently the time-samples in the trial, we extend this method to a generative Bayesian model where each part is formally modelled. This is a main difference compared to the classical approach which usually try to avoid to model the non-informative part. Then, making the assumption that the informative part can be modelled by a time sequence, we adapt the previous method to a Bayesian model of asynchronous template matching which allows the recognition of the time onset of the pattern of interest in each trial. Second, we show the importance of the generative model which, thanks to the Bayesian approach allows us to alleviate the problem of choosing of the hyperparameters of the initial discriminative approach. Compared to the initial discriminant model, us- ing a generative approach leads to use more parameters into the model but whose estimation is helped by the prior we provide. By doing so, we provide a more intuitive way for the experimenter to adapt the method to other problems. Using a generative model and a Bayesian estimation also enables us to improve the generalisation of the model of asynchronous template matching. This model has indeed been tested as benchmark on jittered evoked potential data and has shown to successfully improve the signal-to-noise ratio, recover the evoked response and classify better than classical methods. Finally, we see the importance of asynchronous methods for classification of the EEG correlates of decision-making process. We test this in the context of the study of the exploration/exploitation contrast. Exploration is related to decision making in an uncertain environment. This situation arises a conflict between two opposing needs : gathering information about the environment and exploiting this knowledge in order to optimise the decision. Using an experimental setup that forces the subjects to switch between exploratory or exploitative actions, we show for the first time that it is possible to classify the EEG correlates of the exploratory behaviour. Moreover, we show that synchronous methods fail at classify this contrast thus requiring advanced asynchronous ones. The results also confirm that the brain areas relevant to this switch are mainly the left parietal and medial frontal cortex which is consistent with the neurophysiological findings based on functional magnetic resonance imagery. In addition we have been able to show the importance of alpha rhythm for this contrast. In summary, this thesis provides a formal framework for classification of asynchronous EEG patterns using a generative Bayesian approach. It also provides a methodology to approach the study of EEG correlates of cognitive tasks when little is known about them and when the targeted pattern is reasonably assumed to be non time-locked.

EPFL2012

Bayesian risk analysis of financial time series

Christophe Osinski

This thesis is a contribution to financial statistics. One of the principal concerns of investors is the evaluation of portfolio risk. The notion of risk is vague, but in finance it is always linked to possible losses. In this thesis, we present some measures allowing the valuation of risk with the help of Bayesian methods. An exploratory analysis of data is presented to describe the sampling properties of financial time series. This analysis allows us to understand the origins of the daily returns studied in this thesis. Moreover, a discussion of different models is presented. These models make strong assumptions on investor behaviour, which are not always satisfied. This exploratory analysis shows some differences between the behaviour anticipated under equilibrium models, and that of real data. The Bayesian approach has been chosen because it allows one to incorporate all the variability, in particular that associated with model choice. The models studied in this thesis allow one to take heteroskedasticity into account, as well as particular shapes of the tails of returns. ARCH type models and models based on extreme value theory are studied. One original aspect of this thesis is its use of Bayesian analysis to detect change points in financial time series. We suppose that a market has two phases, and that it switches from a state to the other at random. Another new contribution is a model integrating heteroskedasticity and time dependence of extreme values, by superposition of the model proposed by Bortot and Coles (2003) and a GARCH process. This thesis uses simulation intensively for the estimation of risk measures. The drawback of simulation is the amount of time needed to obtain accurate estimates. However, simulation allows one to produce results when direct calculation is not feasible. For example, simulation allows one to compute risk estimates for time horizons greater than one day. The methods presented in this thesis are illustrated on simulated data, and on real data from European and American markets. This thesis involved the construction of a library containing C and S code to perform risk analysis using GARCH and extreme value theory models. The results show that model uncertainty can be incorporated, and that risk measures for time horizons greater than one can be obtained by simulation. The methods presented in this thesis have a natural representation involving conditioning. Thus, they permit the computation of both conditional and unconditional risk estimates. Three methods are described: the GARCH method; the two-state GARCH method; and the HBC method. Unconditional risk estimation using the GARCH method is satisfactory on data which seem stationary, but not reliable on data which are non-stationary, such as data with change points. The two-state GARCH model does a little better, but gives very satisfactory results when the risk is estimated conditionally on time. The HBC method does not give satisfactory results.