Expressing Motivations By Facilitating Other’s Inverse Reinforcement Learning

It is often necessary to understand each other’s motivations in order to cooperate. Reaching such a mutual understanding requires two abilities: to build models of other’s motivations in order to understand them, and to build a model of “my” motivations perceived by others in order to be understood. Having a self-image seen by others requires two recursive orders of modeling, known in psychology as the first and second orders of theory of mind. In this paper, we present a second-order theory of mind cognitive architecture that aims to facilitate mutual understanding in multi-agent scenarios. We study different conditions of empathy and gratitude leading to irrational cooperation in iterated prisoner’s dilemma.