Essays on individual decision making under uncertainty

This research project is an experimental study of decision-making in very difficult contexts resembling those encountered in financial markets. The starting point was the empirical observation that financial assets are objects of a very complex kind. Specifically, financial assets generate rewards in an unpredictable way (stochastic processes), and they jump regularly (unstable processes). What's more, they are covert: finance practitioners lack information about their nature, and may well be unable to uncover it. After all, can humans really learn the nature of unstable stochastic processes? To address this question, I designed an investment task, the "Boardgame," characterized by assets – in the form of locations displayed on a board – which are, like the financial assets in the real world, unpredictable, unstable, and (initially) unknown. Specifically, the game consists of choosing between six locations of initially unknown probability of winning or losing. Each trial, the player selects one location and immediately receives the outcome returned by the chosen location (a reward, a loss, or 0 CHF). She accumulates gains and losses throughout the game, with the goal of maximizing the cumulative earnings. One essential characteristic of the Boardgame is that outcome probabilities – at all locations – jump regularly. An ideal player accounts for jump occurrence and re-learns outcome probabilities once a jump has occurred. Such two-stage mental process – consisting of jump detection and subsequent reappraisal of the probabilities – is extremely demanding. I invited 62 subjects – all undergraduate or graduate students at the EPFL – to play the Boardgame during 30 minutes, and recorded their action in each trial (namely, which location they chose). From their choices, I attempted to infer the nature of their learning in the game. To do so, I followed an inter-disciplinary approach, at the intersection of economics, machine learning, and neurobiology. I first formalized normative learning in the game (how ideal players learn), as well as bounded rational learning (how adaptive – intelligent – agents, albeit of limited cognitive capabilities, are expected to learn). I then examined whether my subjects did a good job of learning in the Boardgame, by comparing the fits of the normative and bounded rational models. I eventually studied the neural implementation of the behavior at work. It appears that for the majority of my subjects, the normative learning model did a much better job of explaining actual behavior than the alternative, bounded rational model. In other words, subjects were apt at learning the outcome probabilities of the locations, despite the difficulty of the enterprise. This suggests that people may be better fit to cope with the instability of financial markets than previously thought. Beyond being of interest for the finance field, this result has actually a broader scope. For it demonstrates the human ability to adapt behavior to suit an uncertain changing environment. This cognitive flexibility, which some eminent neuroscientists have been taken to be the most relevant definition of human intelligence, is essential in many domains beyond the one studied here (reward learning) – in particular, the same neuroscientists have suggested that it is chief in contexts of social interaction. The finding that my subjects were very good at learning the nature of the locations is also important from an epistemological point of view. For one message drawn from this study is that people can be very sophisticated in the face of very complex problems. This message should prompt a reevaluation of the scope of the dominant paradigm in decision making – namely, the limited cognition paradigm. The limited cognition paradigm has led either to a hyperemphasis on cognitive biases as sources of human mistakes (irrationality), or to the idea that complexity in a cognitive task precludes the emergence of optimal behavior (bounded rationality). In contrast, the present work highlights human adaptation, and further suggests that complexity is not necessarily an obstacle for full rationality to emerge. One reason that my subjects could fare so well in my complex task is that the human brain is fit to cope with the Boardgame, which is reminiscent of the living conditions of the early man. For Homo has long had to deal with an uncertain unstable environment. As a result, the hominoid brain was geared to cope with instability. The finding that optimal behavior prevailed in my experiment prompted me to flesh out the neural implementation of this behavior. In the final stage of this project, I conjectured a neural system that is fit to cope with unforeseen and ever-changing conditions. I attempted to describe this neural system rigorously, in the vein of a recent computational trend in decision neuroscience. This computational approach aims to reveal the neural substrates of subjective states that the behavioral models purport to estimate – e.g., how much reward a subject expected on each trial, how much surprise a subject experienced on each trial, how much uncertainty a subject perceived in her environment on each trial. Likewise, I estimated subjective variables that are essential to implement the optimal Bayesian rule in my task. In particular, I estimated how much credit a subject assigned to the belief that a jump occurred on each trial. This estimation enabled me to make precise predictions as to the neural mechanisms involved in the Boardgame. The incoming step shall be to put these predictions to a test in an imaging study of the Boardgame. As such, the present research has laid the basis for a larger research program to be pursued. It is hoped that it shows the utility of bringing the modeling tools from economics to bear on neurobiological questions. I think that such scientific exchanges carry the possibility of insights useful to build an adequate model of individual decision-making.