Autonomous virtual agents learning a cognitive model and evolving

In this paper, we propose a new integration approach to simulate an autonomous virtual agent's cognitive learning of a task for interactive virtual environment applications. Our research focuses on the behavioural animation of virtual humans capable of acting independently. Our contribution is important because we present a solution for fast learning with evolution. We propose the concept of a learning unit architecture that functions as a control unit of the autonomous virtual agent's brain. Although our technique has, proved to be effective in our case study, there is no guarantee that it will work for every imaginable autonomous virtual agent and virtual environment. The results are illustrated in a domain that requires effective coordination of behaviours, such as driving a car inside a virtual city

Méthodes infographiques pour l'apprentissage des agents autonomes virtuels

Toni Conde

There are two primary approaches to behavioural animation of an Autonomous Virtual Agent (AVA). The first one, or behavioural model, defines how AVA reacts to the current state of its environment. In the second one, or cognitive model, this AVA uses a thought process allowing it to deliberate over its possible actions. Despite the success of these approaches in several domains, there are two notable limitations which we address in this thesis. First, cognitive models are traditionally very slow to execute, as a tree search, in the form of mapping: states → actions, must be performed. On the one hand, an AVA can only make sub-optimal decisions and, on the other hand, the number of AVAs that can be used simultaneously in real-time is limited. These constraints restrict their applications to a small set of candidate actions. Second, cognitive and behavioural models can act unexpectedly, producing undesirable behaviour in certain regions of the state space. This is because it may be impossible to exhaustively test them for the entire state space, especially if the state space is continuous. This can be worrisome for end-user applications involving AVAs, such as training simulators for cars and aeronautics. Our contributions include the design of novel learning methods for approximating behavioural and cognitive models. They address the problem of input selection helped by a novel architecture ALifeE including virtual sensors and perception, regardless of the machine learning technique utilized. The input dimensionality must be kept as small as possible, this is due to the "curse of dimensionality", well known in machine learning. Thus, ALifeE simplifies and speeds up the process for the designer.

EPFL2005

Behavioral animation of autonomous virtual agents helped by reinforcement learning

Toni Conde, Daniel Thalmann

Our research focuses on the behavioral animation of virtual humans who are capable of taking actions by themselves. We deal more specifically with reinforcement learning methodologies, which integrate in an original way the RL agent and the autonomous virtual agent in a virtual environment. With the help of a virtual environment in the form of a town, we demonstrate that it is indeed the learning process and not the optimization of RL, which is used by the AVAs

Springer-Verlag2003

The Learnability of the Grammar of Jazz: Bayesian Inference of Hierarchical Structures in Harmony

Daniel Harasim

Musical grammar describes a set of principles that are used to understand and interpret the structure of a piece according to a musical style. The main topic of this study is grammar induction for harmony --- the process of learning structural principles from the observation of chord sequences. The question how grammars are learnable by induction from sequential data is an instance of the more general question how abstract knowledge is inducible from the observation of data --- a central question of cognitive science. Under the assumption that human learning approximately follows the principles of rational reasoning, Bayesian models of cognition can be used to simulate learning processes. This study investigates what prior knowledge makes it possible to learn musical grammar inductively from Jazz chord sequences using Bayesian models and computational simulations. The theoretical part of the thesis presents how questions about learnability can be studied in a unified framework involving music analysis, cognitive modeling, Bayesian statistics, and computational simulations. A new grammar formalism, called Probabilistic Abstract Context-Free Grammar (PACFG), is proposed that allows for flexible probability models which facilitate the grammar-induction experiments of this study. PACFG can jointly model multiple musical dimensions such as harmony and rhythm, and can use coordinate ascent variational inference for grammar learning. The empirical part of the thesis reports supervised and unsupervised grammar-learning experiments. To train and evaluate grammar models, a ground-truth dataset of hierarchical analyses of complete Jazz standards, called the Jazz Harmony Treebank (JHT), was created. The supervised grammar-learning experiments, in which grammars for Jazz harmony are learned from the JHT analyses, show that jointly modeling harmony and rhythm significantly improves the grammar models' prediction of the ground truth. The performance and robustness of the grammars are further improved by a transpositionally invariant parameterization of rule probabilities. Following the supervised grammar learning, unsupervised grammar learning was performed by inducing harmony grammars merely from Jazz chord sequences, without the observation of the JHT trees. The results show that the best induced grammar performs similarly well as the best supervised grammar. In particular, the goal-directedness of functional harmony does not need to be assumed a priori, but can be learned without usage of music-specific prior knowledge. The findings of this thesis show that general prior knowledge enables an ideal learner to acquire abstract musical principles by statistical learning. In conclusion, it is plausible that much aspects of musical grammar have been learned by Jazz musicians and listeners, instead of being innate predispositions or explicitly taught concepts. This thesis is moreover embedded into the context of empirical music research and digital humanities. Current studies either describe complex musical structures qualitatively or investigate simpler aspects quantitatively. The computational models developed in this thesis demonstrate that deep insights into music and statistical analyses are not mutually exclusive. They enable a new kind of data-driven music theory and musicology, for instance through comparative analyses of musical grammar for different styles such as Jazz, Rock, and Western classical music.

EPFL2020

The Learnability of the Grammar of Jazz: Bayesian Inference of Hierarchical Structures in Harmony

Daniel Harasim

EPFL2020