Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Unités associées (15)

Unités associées

Chargement

Cours enseignés par cette personne (5)

Cours enseignés par cette personne

Chargement

Domaines de recherche associés (94)

Domaines de recherche associés

Chargement

Publications associées (231)

Publications associées

Chargement

Personnes menant des recherches similaires (100)

Personnes menant des recherches similaires

Chargement

Computational models of episodic-like memory in food-caching birds

Johanni Michael Brea, Wulfram Gerstner

How the 'what', 'where', and 'when' of past experiences are stored in episodic memories and retrieved for suitable decisions remains unclear. In an effort to address these questions, the authors present computational models of neural networks that behave like food caching birds in episodic memory tasks. Birds of the crow family adapt food-caching strategies to anticipated needs at the time of cache recovery and rely on memory of the what, where and when of previous caching events to recover their hidden food. It is unclear if this behavior can be explained by simple associative learning or if it relies on higher cognitive processes like mental time-travel. We present a computational model and propose a neural implementation of food-caching behavior. The model has hunger variables for motivational control, reward-modulated update of retrieval and caching policies and an associative neural network for remembering caching events with a memory consolidation mechanism for flexible decoding of the age of a memory. Our methodology of formalizing experimental protocols is transferable to other domains and facilitates model evaluation and experiment design. Here, we show that memory-augmented, associative reinforcement learning without mental time-travel is sufficient to explain the results of 28 behavioral experiments with food-caching birds.

NATURE PORTFOLIO2023

Chargement

A taxonomy of surprise definitions

Johanni Michael Brea, Wulfram Gerstner, Alireza Modirshanechi

Surprising events trigger measurable brain activity and influence human behavior by affecting learning, memory, and decision-making. Currently there is, however, no consensus on the definition of surprise. Here we identify 18 mathematical definitions of surprise in a unifying framework. We first propose a technical classification of these definitions into three groups based on their dependence on an agent’s belief, show how they relate to each other, and prove under what conditions they are indistinguishable. Going beyond this technical analysis, we propose a taxonomy of surprise definitions and classify them into four conceptual categories based on the quantity they measure: (i) ‘prediction surprise’ measures a mismatch between a prediction and an observation; (ii) ‘change-point detection surprise’ measures the probability of a change in the environment; (iii) ‘confidence-corrected surprise’ explicitly accounts for the effect of confidence; and (iv) ‘information gain surprise’ measures the belief-update upon a new observation. The taxonomy poses the foundation for principled studies of the functional roles and physiological signatures of surprise in the brain.

2022

Chargement

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Johanni Michael Brea, Wulfram Gerstner, Marco Philipp Lehmann, Vasiliki Liakoni, Alireza Modirshanechi, Kerstin Preuschoff

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) entanglement of brain signals of different learning modules, and (iii) a limited number of computational models considered as candidates for explaining behavior. Here, we address these three limitations and (i) introduce a complex sequential decision making task with surprising events that allows us to (ii) dissociate correlates of reward prediction errors from those of surprise in functional magnetic resonance imaging (fMRI); and (iii) we test behavior against a large repertoire of model-free, model-based, and hybrid reinforcement learning algorithms, including a novel surprise-modulated actor-critic algorithm. Surprise, derived from an approximate Bayesian approach for learning the world-model, is extracted in our algorithm from a state prediction error. Surprise is then used to modulate the learning rate of a model-free actor, which itself learns via the reward prediction error from model-free value estimation by the critic. We find that action choices are well explained by pure model-free policy gradient, but reaction times and neural data are not. We identify signatures of both model-free and surprise-based learning signals in blood oxygen level dependent (BOLD) responses, supporting the existence of multiple parallel learning modules in the brain. Our results extend previous fMRI findings to a multi-step setting and emphasize the role of policy gradient and surprise signalling in human learning.

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

Chargement

BIO-465: Biological modeling of neural networks

In this course we study mathematical models of neurons and neuronal networks in the context of biology and establish links to models of cognition. The focus is on brain dynamics approximated by determinstic or stochatic differnetial equations.

BIO-623: State of the Art Topics in Neuroscience XII

Surprise, Reward, and Curiosity are drives of human, animal, and robot behavior. The class links theories of reinforcement learning with human behavior beyond standard notions of reward.

BIO-642: State of the Art Topics in Neuroscience XIII

The Loss Landscape of Neural Networks is in general non-convex and rough, but recent mathematical results lead provide insights of practical relevance. 9 online lectures, lecturers from NYU, Stanford, Shanghai, IST Austria, Google, Facebook, EPFL.

Modèles du neurone biologique

vignette|390x390px|Fig. 1. Dendrites, soma et axone myélinisé, avec un flux de signal des entrées aux dendrites aux sorties aux bornes des axones. Le signal est une courte impulsion électrique appelée

Plasticité synaptique

La plasticité synaptique, en neurosciences, désigne la capacité des synapses à moduler, à la suite d'un événement particulier - une augmentation ou une diminution ponctuelle et significative de leur a

Apprentissage

L’apprentissage est un ensemble de mécanismes menant à l'acquisition de savoir-faire, de savoirs ou de connaissances. L'acteur de l'apprentissage est appelé apprenant. On peut opposer l'apprentissag