Stress, individual differences, and norepinephrine in reinforcement learning-based prediction of mouse behavior in conditioning and spatial learning

Acute stress regulates different aspects of behavioral learning through the action of stress hormones and neuromodulators. Stress effects depend on stressor's type, intensity, timing, and the learning paradigm. In addition, genetic background of animals might be an important factor in determining how stress influences their performance. To study these effects I performed conditioning and spatial learning experiments, exposing 2 different genetic strains of mice (C57BL/6 and DBA/2) to a variety of different stressors such as elevated platform, uncertainty in food delivery or in escape platform location, food deprivation, and low water temperature. I first show that these factors have diverse effects on animal behavior in these tasks, often exhibiting nonlinear inter-factor relationships. Mouse behavior in such tasks can be formalized and studied using reinforcement learning (RL) models. In this framework, the effects of stress and genetic background on learning and memory could be attributed to differential dynamics of model meta-parameters (such as learning rate, balance between exploitation and exploration, and future reward discounting), which are thought to be related to the activity of different brain systems, particularly the neuromodulators. In both conditioning and spatial learning experiments, RL models were shown to be flexible enough to reproduce differences in learning and performance that are present between different experimental groups. In order to consider how the combination of modulatory factors such as stress, genetic strain, motivation, anxiety, and manipulations of norepinephrine levels affect animal behavior, I developed a black-box model, which can be trained to use these modulatory factors for determining the suitable meta-parameters of the RL model. Such RL model can successfully predict individual animal behavior in the hole-box conditioning task, given the information about current modulatory factors and animal's previous task performance. In order to study neurobiological correlates, I also performed pharmacological manipulations of adrenergic alpha-2 receptors. Results from these manipulations provide computational insights into how norepinephrine affects performance accuracy and future reward discounting to produce the inverted-U-shape relation between norepinephrine levels and behavioral performance. Such approach makes it possible to understand how genetic strain, acute stress, motivation, uncertainty, and norepinephrine affect learning intensity, performance accuracy, and future reward discounting.