Learning from Failed Demonstrations in Unreliable Systems

This paper presents a method to teach a robot to play Ping Pong from failed demonstrations in a highly noisy and uncertain setting. To infer useful information from failed demonstrations, we use a MultiDonut Algorithm [7] that minimises the probability of repeating a failed demonstration and generates new attempts similar but not quite the same as the demonstration. We compare human demonstrations against a random strategy and show that human demonstrations provide useful information and hence yield faster learning, especially in higher dimensions. We show that learning from observing failed attempts allows the robot to perform the task more reliably than any individual demonstrator did. We also show how this algorithm adapts to gradual deterioration in the system and increases the chances of success when interacting with an unreliable system.

Synthesis, modeling, and experimental validation of distributed robotic search

James Pugh

The work of this thesis centers around the research subject of distributed robotic search. Within the field of distributed robotic systems, a task of particular interest is attempting to locate one or more targets in a possibly unknown environment. While numerous studies have proposed and analyzed methods to accomplish this, there is currently a lack of a strong foundation of tools and techniques that can be used to facilitate the development and evaluation of different approaches to distributed robotic search. In this work, we aim to provide such a foundation through tools, methods, models, and analysis of experimentation. An often overlooked aspect of the distributed robotic research process is the development and analysis of tools and modules to be used with robotic systems. These may include plug-ins for realistic robotic simulators, software/hardware systems to track multiple mobile robots in real-time, extension boards for robotic platforms, and the robots themselves. Along with other tools developed and used for this work, we focus particularly on the development, characterization, and validation of a fast, accurate on-board system for relative positioning and communication between robots, a capability which is critical for effective distributed robotic search. Designing individual robot controllers to generate a specific group behavior is a difficult and often counter-intuitive process. A possible alternative to hand-crafting distributed search controllers is automatic synthesis using machine-learning techniques. We explore the effectiveness of using a noise-resistant version of the Particle Swarm Optimization algorithm to optimize the weights of an embedded Artificial Neural Network, allowing the robot to learn obstacle avoidance behavior (a common benchmark for robotic learning techniques); we find that this technique appears to offer superior performance as compared to the canonical approach of using Genetic Algorithms for this type of learning. A method for faster learning using distributed evaluation in a robot team is tested and is found to offer comparable performance using only a fraction of the original learning time. This technique can be used for fast, effective learning and adaptation in a distributed robotic system performing search. The process of designing and analyzing algorithms for distributed robotic systems can be greatly facilitated if models are available to describe the dynamics of the algorithm at an abstract level. Inspired by previous examples in the distributed robotic field, we work to design a model of robotic search that captures the system at different levels of abstraction, ranging from accurately recreating the details of individual robots to describing the entire system as an indivisible whole. To capture the entire search process, we model both the exploration phase, where robots cover an environment in an effort to detect traces of targets, and the localization phase, where robots use target emission sensing to navigate towards the target. The utility of our models is demonstrated by using them to develop an effective technique for the declaration phase of search, where robots decide that a target has been accurately localized and announce its position. In distributed robotics research, it is important that techniques developed with abstracted simulations and models are ultimately validated using real robotic platforms in order to verify their correctness. In that spirit, we run systematic sound search experiments using teams of up to ten real robots. These experiments utilize the tools developed throughout the research process, demonstrate the utility of our learning technique for fast search adaptation, and serve to validate our models of distributed robotic localization. In addition, they allow us to analyze and better understand the subtle dynamics of the search process, providing information which should be useful for any future work on distributed robotic search.

EPFL2008

Hyper-emotional neurophysiology in a rat model of autism

Mônica Regina Favre

Autism is a pervasive neurodevelopmental disorder of unknown etiology, characterized by a spectrum of symptoms of social impairment, abnormal communication, and unusual restricted, inflexible behaviors. Patients also frequently present various levels of sensory, learning, memory and attention abnormalities, anxiety and phobias, in addition to disturbances in gastrointestinal, sleep, metabolic and immune functions, also with higher rates of physical malformations and seizures. There is currently no empirically based consensus on the causes of symptoms and their remarkable heterogeneity. It is also unknown if associated symptoms are directly linked to autism etiology or if they represent parallel comorbidities. In consequence of our incomplete understanding of the disease process, treatment options are vast and not equally efficacious across all diagnosed individuals. Thus, autism today leads to a lifelong disability that requires significant assistance from health care providers, educational systems and family members. Ultimately, improved diagnostic indicators and personalized therapies should allow the autistic individual to live with reduced costs and more independently. Prenatal exposure to compounds such as valproic acid (VPA) has been associated with higher risk for development of autistic features. The teratogenic and neurodevelopmental effects of VPA in humans are replicated in rodents exposed early in embryonic development, at the time neural tube closure. Thus, the VPA-rat has emerged as a well validated model of autism for controlled investigations of the neurobiological basis of autistic-like symptoms and individual response to environmental interventions. Our laboratory previously demonstrated in the model that the primary sensory cortical, frontal cortical, and amygdala microcircuits are hyper-functional in the pup, in parallel to excessive conditioned fear responses, over-generalization of fear to non-conditioned stimuli, and resistance to fear extinction, in the adult. Together with the full range of autistic symptoms and increasing evidence for local hyper-functional connections and cortical minicolumnpathy in patients, data from the VPA model inspired the hypothesis of autism as an Intense World Syndrome. The theory proposes that the polygenetic background of each individual sets the threshold for susceptibility to their prenatal and post natal environments. At the event of a biological insult, be it exogenous such as VPA-exposure, or genetic, such as a rare set of polymorphisms, a cascade of molecular events would lead to hyper-plastic and hyper-reactive neural microcircuits creating a hyper-functional endophenotype. Depending on the developmental trajectory of each individual, this would lead to abnormalities in four dimensions: hyper-attention, hyper-memory and hyper-perception, and hyper-emotionality. Different patterns of hyper-functional outcomes in each individual would then create a fragmented, intense and even aversive experience of the world, and explain the wide heterogeneity in symptom severity and most, if not all, characteristics of autistic people. Thus, inherent to the theory is the description of autistic individuals as highly biologically sensitive and thus that a predictable, positive, safe and calm environment is an essential feature of early postnatal life for positive outcomes in the child. For my doctoral dissertation, I addressed if embryonic exposure to VPA in the rat leads to hyper-functional emotion system, and if this leads to differential response of individuals to postnatal environment. In Chapter 1, I introduce the clinical characteristics of autism and the brain systems involved in emotion and cognition through modulation of fear, anxiety learning and memory. Then I introduce the research rationale for this project. In Chapter 2, I present our published manuscript describing physical malformations, and other teratogenic and developmental health effects in the VPA-rat. This study further validates the VPA-rat as a model of effects in humans, and summarizes important outstanding questions in the VPA-model, in view of the variability in protocols for model generation. We emphasize that such general health parameters could serve as indicators of the precise time and location of toxic event, and thus as potential predictors of differential symptom dimensions and severities. In Chapter 3, I present our study of predictability in the environment of VPA animals as an essential feature for beneficial effects of enrichment, on autism-like hyper-emotionality. This study provides an elegant and computationally accessible method for multivariate analysis of group effects and individual-based effects. In support of the Intense World Theory, our results suggest that VPA-exposed animals are particularly sensitive to their environment, and point to the hippocampus and amygdala as candidate targets of environmental enrichment predictability effects in reducing fear and anxiety in certain VPA exposed individuals. Finally, in Chapter 4, I present some concluding remarks on the implications of this project and future perspectives.

EPFL2013

Open-ended Learning of Visual and Multi-modal Patterns

Jie Luo

A common trend in machine learning and pattern classification research is the exploitation of massive amounts of information in order to achieve an increase in performance. In particular, learning from huge collections of data obtained from the web, and using multiple features generated from different sources, have led to significantly boost of performance on problems that have been considered very hard for several years. In this thesis, we present two ways of using these information to build learning systems with robust performance and some degrees of autonomy. These ways are Cue Integration and Cue Exploitation, and constitute the two building blocks of this thesis. In the first block, we introduce several algorithms to answer the research question on how to integrate optimally multiple features. We first present a simple online learning framework which is a wrapper algorithm based on the high-level integration approach in the cue integration literature. It can be implemented with existing online learning algorithms, and preserves the theoretical properties of the algorithms being used. We then extend the Multiple Kernel Learning (MKL) framework, where each feature is converted into a kernel and the system learns the cue integration classifier by solving a joint optimization problem. To make the problem practical, We have designed two new regularization functions making it possible to optimize the problem efficiently. This results in the first online method for MKL. We also show two algorithms to solve the batch problem of MKL. Both of them have a guaranteed convergence rate. These approaches achieve state-of-the-art performance on several standard benchmark datasets, and are order of magnitude faster than other MKL solvers. In the second block, We present two examples on how to exploit information between different sources, in order to reduce the effort of labeling a large amount of training data. The first example is an algorithm to learn from partially annotated data, where each data point is tagged with a few possible labels. We show that it is possible to train a face classification system from data gathered from Internet, without any human labeling, but generating in an automatic way possible lists of labels from the captions of the images. Another example is under the transfer learning setting. The system uses existing models from potentially correlated tasks as experts, and transfers their outputs over the new incoming samples, of a new learning task where very few labeled data are available, to boost the performance.

Ecole polytechnique fédérale de Lausanne2011