Ageing and reinforcement learning

Learning to achieve one’s goal in a complex environment is a complicated task. In reinforcement learning (RL) tasks, an agent interacts with the environment to learn optimal actions. In humans, striatal areas are strongly involved in these tasks. During aging, neurotransmitter levels in these regions decrease provoking the question of how RL changes. Here, we developed a RL paradigm comprised of several different states. For each state, there were four actions, which brought the participants to another state. The objective of the participants was to reach a pre-defined goal state as many times as possible in 8 (short inter-stimulus interval condition) and 40 (long inter-stimulus interval condition) minutes. We tested a cohort of 40 healthy older participants (mean age = 68.75 ± 8.24) and 30 healthy young adults (mean age = 25.03 ± 4.09). Young participants reached more goals and made more correct actions, within the given time limit, than the ageing cohort. In addition, older participants exhibited more perseverative behaviors, namely, they repeatedly performed specific state-action pairs regardless of the correctness. By applying a Q-learning model, we observed a significant difference between older and young people in the exploration rate but not in the learning rate and forgetting rate. From the computation model, we demonstrated that the deteriorated performance for ageing is due to the deficit in neither learning nor working memory but from more suboptimal judgments of the action selections. These results are in line with the studies that less responsiveness of the striatal areas leads to more perseverative actions.