A learning by imitation model handling multiple constraints and motion alternatives

We present a probabilistic approach to learn robust models of human motion through imitation. The association of Hidden Markov Model (HMM), Gaussian Mixture Regression (GMR) and dynamical systems allows us to extract redundancies across multiple demonstrations and build time-independent models to reproduce the dynamics of the demonstrated movements. The approach is first systematically evaluated and compared with other approaches by using generated trajectories sharing similarities with human gestures. Three applications on different types of robots are then presented. An experiment with the iCub humanoid robot acquiring a bimanual dancing motion is first presented to show that the system can also handle cyclic motion. An experiment with a 7 DOFs WAM robotic arm learning the motion of hitting a ball with a table tennis racket is presented to highlight the possibility to encode several variations of a movement in a single model. Finally, an experiment with a HOAP-3 humanoid robot learning to manipulate a spoon to feed the Robota humanoid robot is presented to demonstrate the capability of the system to handle several constraints simultaneously.

Learning and reproduction of gestures by imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression

Aude Billard, Sylvain Calinon, Eric Sauser

Institute of Electrical and Electronics Engineers2010

Continuous extraction of task constraints in a robot programming by demonstration framework

Sylvain Calinon

Robot Programming by Demonstration (RbD), also referred to as Learning by Imitation, explores user-friendly means of teaching a robot new skills. Recent advances in RbD have identified a number of key-issues for ensuring a generic approach to the transfer of skills across various agents and contexts. This thesis focuses on the two generic questions of what-to-imitate and how-to-imitate, which are respectively concerned with the problem of extracting the essential features of a task and determining a way to reproduce these essential features in different situations. The perspective adopted in this work is that a skill can be described efficiently at a trajectory level and that the robot may infer what are the important characteristics of the skill by observing multiple demonstrations of it, assuming that the important characteristics are invariant across the demonstrations. The proposed approach is based on the use of well-established statistical methods in a RbD application, by using Hidden Markov Model (HMM) as a first approach and then moving on to the joint use of Gaussian Mixture Model (GMM) and Gaussian Mixture Regression (GMR). Even if these methods were applied extensively in various fields of research including human motion analysis and robotics, their use essentially focused on gesture recognition rather than on gesture reproduction. Moreover, the models were usually trained offline using large datasets. Thus, the use and combination of these machine learning techniques in a Learning by Imitation framework is challenging and has not been extensively studied yet. In this thesis, we show that these methods are well suited for incremental and active teaching scenarios where the user only provides a small number of demonstrations, either by wearing a set of motions sensors attached to his/her body, or by helping the robot refine its skill by kinesthetic teaching, that is, by embodying the robot and putting it through the motion. These techniques are then applied for enabling a Fujitsu HOAP-2 and HOAP-3 humanoid robot to extract automatically different constraints by observing several manipulation skills and to reproduce these skills in various situations through Gaussian Mixture Regression (GMR). The contributions of this thesis are threefold: (1) it contributes to RbD by proposing a generic probabilistic framework to deal with recognition, generalization, reproduction and evaluation issues, and more specifically to deal with the automatic extraction of task constraints and with the determination of a controller satisfying several constraints simultaneously to reproduce the skill in a different context; (2) it contributes to Human-Robot Interaction (HRI) by proposing active teaching methods that puts the human teacher "in the loop" of the robot's learning by using an incremental scaffolding process and by using different modalities to produce the demonstrations; (3) it finally contributes to robotics and HRI through various real-world applications of the framework showing learning and reproduction of communicative gestures and manipulation skills. The generality of the proposed framework is also demonstrated through its joint use with other learning approaches in collaborative experiments.

EPFL2007

Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework

Aude Billard, Sylvain Calinon

We consider the problem of learning robust models of robot motion through demonstration. An approach based on Hidden Markov Model (HMM) and Gaussian Mixture Regression (GMR) is proposed to extract redundancies across multiple demonstrations, and build a time- independent model of a set of movements demonstrated by a human user. Two experiments are presented to validate the method, that consist of learning to hit a ball with a robotic arm, and of teaching a humanoid robot to manipulate a spoon to feed another humanoid. The experiments demonstrate that the proposed model can efficiently handle several aspects of learning by imitation. We first show that it can be utilized in an unsupervised learning manner, where the robot is autonomously organizing and encoding variants of motion from the multiple demonstrations. We then show that the approach allows to robustly generalize the observed skill by taking into account multiple constraints in task space during reproduction.

2009

Continuous extraction of task constraints in a robot programming by demonstration framework

Sylvain Calinon

EPFL2007