Home

Person

Wulfram Gerstner

Related publications (129)

About
Privacy
Disclaimer

Graph Chatbot

Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances

Wulfram Gerstner, Clément Hongler, Johanni Michael Brea, Francesco Spadaro, Berfin Simsek, Arthur Jacot

We study how permutation symmetries in overparameterized multi-layer neural networks generate `symmetry-induced' critical points. Assuming a network with

L

layers of minimal widths

r_1^*, \ldots, r_{L-1}^*

reaches a zero-loss minimum at $ r_1^*! \c ...

2021

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

Wulfram Gerstner, Jean Ventura, Bernd Albert Illing

Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances ...

2021

When shared concept cells support associations: Theory of overlapping memory engrams

Wulfram Gerstner, Emanuela De Falco, Tilo Schwalger, Chiara Gastaldi

Assemblies of neurons, called concepts cells, encode acquired concepts in human Medial Temporal Lobe. Those concept cells that are shared between two assemblies have been hypothesized to encode associations between concepts. Here we test this hypothesis in ...

2021

Rapid suppression and sustained activation of distinct cortical regions for a delayed sensory-triggered motor response

Wulfram Gerstner, Carl Petersen, Sylvain Crochet, Alireza Modirshanechi, Vahid Esmaeili, Georgios Foustoukos, Samuel Pavio Muscinelli, Yanqi Liu, Keita Tamura, Anastasiia Oryshchuk

The neuronal mechanisms generating a delayed motor response initiated by a sensory cue remain elusive. Here, we tracked the precise sequence of cortical activity in mice transforming a brief whisker stimulus into delayed licking using wide-field calcium im ...

2021

A functional model of adult dentate gyrus neurogenesis

Wulfram Gerstner, Olivia Gozel

In adult dentate gyrus neurogenesis, the link between maturation of newborn neurons and their function, such as behavioral pattern separation, has remained puzzling. By analyzing a theoretical model, we show that the switch from excitation to inhibition of ...

2021

Learning in Volatile Environments With the Bayes Factor Surprise

Wulfram Gerstner, Johanni Michael Brea, Alireza Modirshanechi, Vasiliki Liakoni

Surprise-based learning allows agents to rapidly adapt to nonstationary stochastic environments characterized by sudden changes. We show that exact Bayesian inference in a hierarchical model gives rise to a surprise-modulated trade-off between forgetting o ...

MIT PRESS2021

Mesoscopic population equations for spiking neural networks with synaptic short-term plasticity

Wulfram Gerstner, Tilo Schwalger, Valentin Marc Schmutz

Coarse-graining microscopic models of biological neural networks to obtain mesoscopic models of neural activities is an essential step towards multi-scale models of the brain. Here, we extend a recent theory for mesoscopic population dynamics with static s ...

2020

Dendritic Voltage Recordings Explain Paradoxical Synaptic Plasticity: A Modeling Study

Wulfram Gerstner, Matthias Chinyen Tsai, Laureline Logiaco, Claire Meissner-Bernard

Experiments have shown that the same stimulation pattern that causes Long-Term Potentiation in proximal synapses, will induce Long-Term Depression in distal ones. In order to understand these, and other, surprising observations we use a phenomenological mo ...

FRONTIERS RESEARCH FOUNDATION2020

On the choice of metric in gradient-based theories of brain function

Wulfram Gerstner, Johanni Michael Brea, Jean-Pascal Théodor Pfister

The idea that the brain functions so as to minimize certain costs pervades theoretical neuroscience. Because a cost function by itself does not predict how the brain finds its minima, additional assumptions about the optimization method need to be made to ...

2020

Biologically plausible deep learning – but how far can we go with shallow networks?

Wulfram Gerstner, Johanni Michael Brea, Bernd Albert Illing

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining s ...

2019

Page 3 of 13

Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances

Wulfram Gerstner, Clément Hongler, Johanni Michael Brea, Francesco Spadaro, Berfin Simsek, Arthur Jacot

We study how permutation symmetries in overparameterized multi-layer neural networks generate `symmetry-induced' critical points. Assuming a network with

L

layers of minimal widths

r_1^*, \ldots, r_{L-1}^*

reaches a zero-loss minimum at $ r_1^*! \c ...

2021