Are you an EPFL student looking for a semester project?

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Graph
Search

fr|en

Unit

Laboratoire de calcul neuromimetique (SV/IC)

Laboratory

Related publications (96)

A taxonomy of surprise definitions

Johanni Michael Brea, Wulfram Gerstner, Alireza Modirshanechi

Surprising events trigger measurable brain activity and influence human behavior by affecting learning, memory, and decision-making. Currently there is, however, no consensus on the definition of surprise. Here we identify 18 mathematical definitions of surprise in a unifying framework. We first propose a technical classification of these definitions into three groups based on their dependence on an agent’s belief, show how they relate to each other, and prove under what conditions they are indistinguishable. Going beyond this technical analysis, we propose a taxonomy of surprise definitions and classify them into four conceptual categories based on the quantity they measure: (i) ‘prediction surprise’ measures a mismatch between a prediction and an observation; (ii) ‘change-point detection surprise’ measures the probability of a change in the environment; (iii) ‘confidence-corrected surprise’ explicitly accounts for the effect of confidence; and (iv) ‘information gain surprise’ measures the belief-update upon a new observation. The taxonomy poses the foundation for principled studies of the functional roles and physiological signatures of surprise in the brain.

2022

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Johanni Michael Brea, Wulfram Gerstner, Marco Philipp Lehmann, Vasiliki Liakoni, Alireza Modirshanechi, Kerstin Preuschoff

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) entanglement of brain signals of different learning modules, and (iii) a limited number of computational models considered as candidates for explaining behavior. Here, we address these three limitations and (i) introduce a complex sequential decision making task with surprising events that allows us to (ii) dissociate correlates of reward prediction errors from those of surprise in functional magnetic resonance imaging (fMRI); and (iii) we test behavior against a large repertoire of model-free, model-based, and hybrid reinforcement learning algorithms, including a novel surprise-modulated actor-critic algorithm. Surprise, derived from an approximate Bayesian approach for learning the world-model, is extracted in our algorithm from a state prediction error. Surprise is then used to modulate the learning rate of a model-free actor, which itself learns via the reward prediction error from model-free value estimation by the critic. We find that action choices are well explained by pure model-free policy gradient, but reaction times and neural data are not. We identify signatures of both model-free and surprise-based learning signals in blood oxygen level dependent (BOLD) responses, supporting the existence of multiple parallel learning modules in the brain. Our results extend previous fMRI findings to a multi-step setting and emphasize the role of policy gradient and surprise signalling in human learning.

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances

Johanni Michael Brea, Wulfram Gerstner, Clément Hongler, Berfin Simsek, Francesco Spadaro

We study how permutation symmetries in overparameterized multi-layer neural networks generate `symmetry-induced' critical points. Assuming a network with

L

layers of minimal widths

r_1^*, \ldots, r_{L-1}^*

reaches a zero-loss minimum at

r_1^*! \cdots r_{L-1}^*!

isolated points that are permutations of one another, we show that adding one extra neuron to each layer is sufficient to connect all these previously discrete minima into a single manifold. For a two-layer overparameterized network of width

r^*+ h =: m

we explicitly describe the manifold of global minima: it consists of

T(r^*, m)

affine subspaces of dimension at least

h

that are connected to one another. For a network of width

m

, we identify the number

G(r,m)

of affine subspaces containing only symmetry-induced critical points that are related to the critical points of a smaller network of width

r&lt;r^*

. Via a combinatorial analysis, we derive closed-form formulas for

T

and

G

and show that the number of symmetry-induced critical subspaces dominates the number of affine subspaces forming the global minima manifold in the mildly overparameterized regime (small

h

) and vice versa in the vastly overparameterized regime (

h \gg r^*

). Our results provide new insights into the minimization of the non-convex loss function of overparameterized neural networks.

2021

Fitting summary statistics of neural data with a differentiable spiking network simulator

Johanni Michael Brea, Wulfram Gerstner, Alireza Modirshanechi, Shuqi Wang

Fitting network models to neural activity is an important tool in neuroscience. A popular approach is to model a brain area with a probabilistic recurrent spiking network whose parameters maximize the likelihood of the recorded activity. Although this is widely used, we show that the resulting model does not produce realistic neural activity. To correct for this, we suggest to augment the log-likelihood with terms that measure the dissimilarity between simulated and recorded activity. This dissimilarity is defined via summary statistics commonly used in neuroscience and the optimization is efficient because it relies on back-propagation through the stochastically simulated spike trains. We analyze this method theoretically and show empirically that it generates more realistic activity statistics. We find that it improves upon other fitting algorithms for spiking network models like GLMs (Generalized Linear Models) which do not usually rely on back-propagation. This new fitting algorithm also enables the consideration of hidden neurons which is otherwise notoriously hard, and we show that it can be crucial when trying to infer the network connectivity from spike recordings.

2021

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

Wulfram Gerstner, Bernd Albert Illing, Jean Ventura

Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes a simple layer-specific loss function and does not need to back-propagate error signals within or between layers. Instead, weight updates follow a local, Hebbian, learning rule that only depends on pre-and post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups of neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (ie rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video.

2021

When shared concept cells support associations: Theory of overlapping memory engrams

Emanuela De Falco, Chiara Gastaldi, Wulfram Gerstner, Tilo Schwalger

Assemblies of neurons, called concepts cells, encode acquired concepts in human Medial Temporal Lobe. Those concept cells that are shared between two assemblies have been hypothesized to encode associations between concepts. Here we test this hypothesis in a computational model of attractor neural networks. We find that for concepts encoded in sparse neural assemblies there is a minimal fraction cmin of neurons shared between assemblies below which associations cannot be reliably implemented; and a maximal fraction cmax of shared neurons above which single concepts can no longer be retrieved. In the presence of a periodically modulated background signal, such as hippocampal oscillations, recall takes the form of association chains reminiscent of those postulated by theories of free recall of words. Predictions of an iterative overlap-generating model match experimental data on the number of concepts to which a neuron responds.

2021

A Laplacian pyramid scheme in graph signal processing

Mohammadjavad Faraji

In this report we extend the ideas behind classical multiscale signal processing techniques in order to analyze data residing on graphs. In particular, we extend the notions of filtering, downsampling, and upsampling to functions defined on graphs. We then use these notions to define a Laplacian pyramid scheme that generates a multiscale transform for signals on graphs. Possible applications of our proposed transform include coding, denoising, and function recovery which are among the most important tasks in signal processing.

2011

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biologically plausible models of unsupervised learning of hierarchical representations as found in the brain and modern computer vision models. We use computational modeling to address three main questions at the intersection of artificial intelligence (AI) and computational neuroscience.The first question is: What are useful neural representations and when are deep hierarchical representations needed? We approach this point with a systematic study of biologically plausible unsupervised feature learning in a shallow 2-layer networks on digit (MNIST) and object (CIFAR10) classification. Surprisingly, random features support high performance, especially for large hidden layers. When combined with localized receptive fields, random feature networks approach the performance of supervised backpropagation on MNIST, but not on CIFAR10. We suggest that future models of biologically plausible learning should outperform such random feature benchmarks on MNIST, or that such models should be evaluated in different ways.The second question is: How can hierarchical representations be learned with mechanisms supported by neuroscientific evidence? We cover this question by proposing a unifying Hebbian model, inspired by common models of V1 simple and complex cells based on unsupervised sparse coding and temporal invariance learning. In shallow 2-layer networks, our model reproduces learning of simple and complex cell receptive fields, as found in V1. In deeper networks, we stack multiple layers of Hebbian learning but find that it does not yield hierarchical representations of increasing usefulness. From this, we hypothesise that standard Hebbian rules are too constrained to build increasingly useful representations, as observed in higher areas of the visual cortex or deep artificial neural networks.The third question is: Can AI inspire learning models that build deep representations and are still biologically plausible? We address this question by proposing a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. The proposed rule is Hebbian, i.e. only depends on pre- and post-synaptic neuronal activity, but includes additional local factors, namely predictive dendritic input and widely broadcasted modulation factors. Algorithmically, this rule applies self-supervised contrastive predictive learning to a causal, biological setting using saccades. We find that networks trained with this generalised Hebbian rule build deep hierarchical representations of images, speech and video.We see our modeling as a potential starting point for both, new hypotheses, that can be tested experimentally, and novel AI models that could benefit from added biological realism.

EPFL2021

Search for CP violation in Xi(-)(b) -> pK(-)K(-) decays

Hossein Afsharnia, Liupan An, Vladislav Balagura, Aurelio Bay, Violaine Bellée, Federico Betti, Frédéric Blanc, Alexey Boldyrev, Maxim Borisyak, Sara Celani, Chunhui Chen, Serhii Cholak, Peter Clarke, Victor Coco, Tommaso Colombo, Adam Davis, Michel De Cian, Hans Dijkstra, Paolo Durante, Dipanwita Dutta, Surapat Ek-In, François Fleuret, Christoph Frei, Elena Graverini, Marco Guarise, Guido Haefeli, Xiaoxue Han, Maxim Karpov, Veronica Sølund Kirsebom, Almagul Kondybayeva, Yiming Li, Ho Ling Li, Hao Liu, Shuai Liu, Vladimir Macko, Maurizio Martinelli, Simone Meloni, Tatsuya Nakada, Tara Nanut, Matthew Needham, Niko Neufeld, Preema Rennee Pais, Anton Petrov, Guillaume Max Pietrzyk, Renato Quagliani, Gerhard Raven, Federico Leo Redi, Olivier Schneider, Maxime Schubiger, Sebastian Schulte, Lesya Shchutska, Evgenii Shmanin, Pavol Stefko, Maria Elena Stramaglia, Liang Sun, Frédéric Teubert, Mark Tobin, Minh Tâm Tran, Carina Trippl, Yi Wang, Rui Wang, Mingkui Wang, Zheng Wang, Jian Wang, Zhirui Xu, Yong Yang, Hang Yin, Ettore Zaffaroni, Lei Zhang, Yi Zhang, Yu Zheng, Yao Zhou, Xiaoqing Zhou

A search for CP violation in charmless three-body Xi(-)(b) -> pK(-)K(-) decays is performed using pp collision data recorded with the LHCb detector, corresponding to integrated luminosities of 1 fb(-1) at a center-of-mass energy root S = 7 TeV, 2 fb(-1) at root S = 8 TeV and 2 fb(-1) at = 13 TeV. A good description of the phase-space distribution is obtained with an amplitude model containing contributions from Sigma(1385), Lambda(1405), Lambda(1520), Lambda(1670), Sigma(1775) and Sigma(1915) resonances. The model allows for CP violation effects, which are found to be consistent with zero. The branching fractions of Xi(-)(b) -> Sigma(1385)K-, Xi(-)(b) -> Lambda(1405)K-, Xi(-)(b) -> Lambda(1520)K-, Xi(-)(b) -> Lambda(1670)K-, Xi(-)(b) -> Sigma(1775)K- and Xi(-)(b) -> Sigma(1915)K- decays arc also reported. In addition, an upper limit is placed on the product of ratios of Omega(-)(b) and Xi(-)(b) fragmentation fractions and the Omega(-)(b) -> pK(-)K(-) and Xi(-)(b) -> pK(-)K(-) branching fractions.

AMER PHYSICAL SOC2021

Branching Fraction Measurements of the Rare B-s(0) -> phi mu(+)mu(-) and B-s(0)-> f(2)' (1525)mu(+)mu(-) Decays

The branching fraction of the rare B-s(0) -> phi mu(+)mu(-) decay is measured using data collected by the LHCb experiment at center-of-mass energies of 7, 8, and 13 TeV, corresponding to integrated luminosities of 1, 2, and 6 fb(-1), respectively. The branching fraction is reported in intervals of q(2), the square of the dimuon invariant mass. In the q(2) region between 1.1 and 6.0 GeV2/c(4) , the measurement is found to lie 3.6 standard deviations below a standard model prediction based on a combination of light cone sum rule and lattice QCD calculations. In addition, the first observation of the rare B-s(0)-> f(2)' (1525)mu(+)mu(-) decay is reported with a statistical significance of 9 standard deviations and its branching fraction is determined.

AMER PHYSICAL SOC2021

Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances

Johanni Michael Brea, Wulfram Gerstner, Clément Hongler, Berfin Simsek, Francesco Spadaro

We study how permutation symmetries in overparameterized multi-layer neural networks generate `symmetry-induced' critical points. Assuming a network with

L

layers of minimal widths

r_1^*, \ldots, r_{L-1}^*

reaches a zero-loss minimum at

r_1^*! \cdots r_{L-1}^*!

r^*+ h =: m

we explicitly describe the manifold of global minima: it consists of

T(r^*, m)

affine subspaces of dimension at least

h

that are connected to one another. For a network of width

m

, we identify the number

G(r,m)

of affine subspaces containing only symmetry-induced critical points that are related to the critical points of a smaller network of width

r&lt;r^*

. Via a combinatorial analysis, we derive closed-form formulas for

T

and

G

and show that the number of symmetry-induced critical subspaces dominates the number of affine subspaces forming the global minima manifold in the mildly overparameterized regime (small

h

) and vice versa in the vastly overparameterized regime (

h \gg r^*

). Our results provide new insights into the minimization of the non-convex loss function of overparameterized neural networks.

2021