Publications by He Xu | EPFL Graph Search

Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making

Michael Herzog, Wulfram Gerstner, Alireza Modirshanechi, Marco Philipp Lehmann, He Xu

Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To ...

2021

Neural Correlates of Reinforcement Learning: eligibility trace, reward prediction error, novelty and surprise

He Xu

In reinforcement learning (RL), an agent makes sequential decisions to maximise the reward it can obtain from an environment. During learning, the actual and expected outcomes are compared to tell whether a decision was good or bad. The difference between ...

EPFL2020

One-shot learning and behavioral eligibility traces in sequential decision making

Michael Herzog, Wulfram Gerstner, Marco Philipp Lehmann, Kerstin Preuschoff, He Xu, Vasiliki Liakoni

In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms s ...

2019

Time course of prediction errors in sequential decision making

Michael Herzog, He Xu

In reinforcement learning, an agent makes sequential decisions to maximize reward. During learning, the actual and expected outcome are compared to tell whether a decision was good or bad. The difference between the actual outcome and expected outcome is t ...

2018

Sequential Decision Making: From Vision to Decisions and Back

Michael Herzog, He Xu

Association for Research in Vision and Ophthalmology2017

Neural Correlates of Sequential Decision Making

Michael Herzog, He Xu

In chess, a series of moves is made until a delayed sparse feedback (win, loss) is issued, which makes it impossible to evaluate the value of a single move. There are powerful reinforcement learning (RL) algorithms, which can cope with these sequential dec ...

2017

Neural Correlates of Sequential Decision Making

Michael Herzog, He Xu

In chess, a series of moves is made until a delayed sparse feedback (win, loss) is issued, which makes it impossible to evaluate the value of a single move. There are powerful reinforcement learning (RL) algorithms, which can cope with these sequential dec ...

2017

Reinforcement learning: the effect of environment

Michael Herzog, He Xu

Reinforcement learning is a type of supervised learning, where reward is sparse and delayed. For example in chess, a series of moves is made until a sparse reward (win, loss) is issued, which makes it impossible to evaluate the value of a single move. Stil ...

2016

Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Touradj Ebrahimi, Martin Rerabek, Eleni Kroupi, Anne-Flore Nicole Marie Perrin, He Xu

This paper presents a novel multimodal dataset for the analysis of Quality of Experience (QoE) in emerging immersive multimedia technologies. In particular, the perceived Sense of Presence (SoP) induced by one-minute long video stimuli is explored with res ...

2015