A Dynamical System Approach for Adaptive Grasping, Navigation and Co-Manipulation with Humanoid Robots

In this paper, we present an integrated approach that provides compliant control of an iCub humanoid robot and adaptive reaching, grasping, navigating and co-manipulating capabilities. We use state-dependent dynamical systems (DS) to (i) coordinate and drive the robots hands (in both position and orientation) to grasp an object using an intermediate virtual object, and (ii) drive the robot’s base while walking/navigating. The use of DS as motion generators allows us to adapt smoothly as the object moves and to re-plan on-line motion of the arms and body to reach the object’s new location. The desired motion generated by the DS are used in combination with a whole-body compliant control strategy that absorbs perturbations while walking and offers compliant behaviors for grasping and manipulation tasks. Further, the desired dynamics for the arm and body can be learned from demonstrations. By integrating these components, we achieve unprecedented adaptive behaviors for whole body manipulation. We showcase this in simulations and real-world experiments where iCub robots (i) walk-to-grasp objects, (ii) follow a human (or another iCub) through interaction and (iii) learn to navigate or co-manipulate an object from human guided demonstrations; whilst being robust to changing targets and perturbations.

Imitation Learning of Motion Coordination in Robots

Elena Gribovskaya

The ease with which humans coordinate all their limbs is fascinating. Such a simplicity is the result of a complex process of motor coordination, i.e. the ability to resolve the biomechanical redundancy in an efficient and repeatable manner. Coordination enables a wide variety of everyday human activities from filling in a glass with water to pair figure skating. Therefore, it is highly desirable to endow robots with similar skills. Despite the apparent diversity of coordinated motions, all of them share a crucial similarity: these motions are dictated by underlying constraints. The constraints shape the formation of the coordination patterns between the different degrees of freedom. Coordination constraints may take a spatio-temporal form; for instance, during bimanual object reaching or while catching a ball on the fly. They also may relate to the dynamics of the task; for instance, when one applies a specific force profile to carry a load. In this thesis, we develop a framework for teaching coordination skills to robots. Coordination may take different forms, here, we focus on teaching a robot intra-limb and bimanual coordination, as well as coordination with a human during physical collaborative tasks. We use tools from well-established domains of Bayesian semiparametric learning (Gaussian Mixture Models and Regression, Hidden Markov Models), nonlinear dynamics, and adaptive control. We take a biologically inspired approach to robot control. Specifically, we adopt an imitation learning perspective to skill transfer, that offers a seamless and intuitive way of capturing the constraints contained in natural human movements. As the robot is taught from motion data provided by a human teacher, we exploit evidence from human motor control of the temporal evolution of human motions that may be described by dynamical systems. Throughout this thesis, we demonstrate that the dynamical system view on movement formation facilitates coordination control in robots. We explain how our framework for teaching coordination to a robot is built up, starting from intra-limb coordination and control, moving to bimanual coordination, and finally to physical interaction with a human. The dissertation opens with the discussion of learning discrete task-level coordination patterns, such as spatio-temporal constraints emerging between the two arms in bimanual manipulation tasks. The encoding of bimanual constraints occurs at the task level and proceeds through a discretization of the task as sequences of bimanual constraints. Once the constraints are learned, the robot utilizes them to couple the two dynamical systems that generate kinematic trajectories for the hands. Explicit coupling of the dynamical systems ensures accurate reproduction of the learned constraints, and proves to be crucial for successful accomplishment of the task. In the second part of this thesis, we consider learning one-arm control policies. We present an approach to extracting non-linear autonomous dynamical systems from kinematic data of arbitrary point-to-point motions. The proposed method aims to tackle the fundamental questions of learning robot coordination: (i) how to infer a motion representation that captures a multivariate coordination pattern between degrees of freedom and that generalizes this pattern to unseen contexts; (ii) whether the policy learned directly from demonstrations can provide robustness against spatial and temporal perturbations. Finally, we demonstrate that the developed dynamical system approach to coordination may go beyond kinematic motion learning. We consider physical interactions between a robot and a human in situations where they jointly perform manipulation tasks; in particular, the problem of collaborative carrying and positioning of a load. We extend the approach proposed in the second part of this thesis to incorporate haptic information into the learning process. As a result, the robot adapts its kinematic motion plan according to human intentions expressed through the haptic signals. Even after the robot has learned the task model, the human still remains a complex contact environment. To ensure robustness of the robot behavior in the face of the variability inherent to human movements, we wrap the learned task model in an adaptive impedance controller with automatic gain tuning. The techniques, developed in this thesis, have been applied to enable learning of unimanual and bimanual manipulation tasks on the robotics platforms HOAP-3, KATANA, and i-Cub, as well as to endow a pair of simulated robots with the ability to perform a manipulation task in the physical collaboration.

EPFL2012

From High-Level to Low-Level Robot Learning of Complex Tasks: Leveraging Priors, Metrics and Dynamical Systems

Nadia Barbara Figueroa Fernandez

Humans have a remarkable way of learning, adapting and mastering new manipulation tasks. With the current advances in Machine Learning (ML), the promise of having robots with such capabilities seems to be on the cusp of reality. Transferring human-level skills to robots, however, is complicated as they involve a level of complexity that cannot be tackled by classical ML methods in an unsupervised way. Such complexities involve: (i) automatically decomposing tasks into control-oriented encodings, (ii) extracting invariances and handling idiosyncrasies of data acquired from human demonstrations, and (iii) learning models that guarantee stability and convergence. In this thesis, we push the boundaries in the learning from demonstration (LfD) domain by addressing these challenges with minimal human intervention and parameter tuning. We introduce novel learning approaches based on Bayesian non-parametrics and kernel-methods while encoding novel metrics and priors, leveraging them with dynamical systems (DS) theory. In the first part of this thesis we focus on learning complex sequential manipulation tasks from heterogeneous and unstructured demonstrations. Such tasks are those composed of a sequence of discrete actions. The particular challenge is learning these tasks without any prior knowledge on the number of actions (unstructuredness) or restrictions as to how the human is demonstrating the task (heterogeneous), e.g. changes in reference frames. We propose a Bayesian non-parametric learning framework that can jointly segment and discover unique discrete actions (and their sequence) in continuous demonstrations of the task. Hence, we learn an entire task from a continuous, unrestricted natural demonstration. The learned structure of the complex tasks and the segmented data were then used to parametrize a hybrid controller to execute two cooking tasks of increasing complexity: (i) single-arm pizza dough rolling, and (ii) a dual-arm vegetable peeling task. Throughout this thesis, we assume that both the human and robot motions are driven by autonomous state-dependent DS. Hence, in the second part of this thesis we offer two novel DS formulations to represent and execute a complex task. We begin by proposing a DS-based motion generator formulation and learning scheme that is capable of automatically encoding continuous and complex motions while ensuring global asymptotic stability. The type of tasks that can be learned with this approach are unparalleled to previous work in DS-based LfD and are validated on production line and household activities. Further, we propose a novel DS formulation and learning scheme that can encode both complex motions and varying impedance requirements along the task; i.e., the robot must be compliant in some regions of the task, while stiff in others. This approach is validated on trajectory tracking tasks where a robot arm must precisely draw letters on a surface. Due to the generalization power and straight-forward learning schemes of the proposed DS-based motion generators, we also apply them to more complex application. Such applications include providing adaptive (i) navigation strategies for mobile agents, and (ii) locomotion and co-manipulation tasks of biped robots. These applications are particularly novel in the LfD domain, as most work is solely focused on robotic arm control. In the last part of the thesis, we explore learning complex behaviors in joint space for single and multi-arm systems.

EPFL2019

Adaptive sensorimotor peripersonal space representation and motor learning for a humanoid robot

Micha Hersch

This thesis presents possible computational mechanisms by which a humanoid robot can develop a coherent representation of the space within its reach (its peripersonal space), and use it to control its movements. Those mechanisms are inspired by current theories of peripersonal space representation and motor control in humans, targeting a cross-fertilization between robotics on one side, and cognitive science on the other side. This research addresses the issue of adaptivity the sensorimotor level, at the control level and at the level of simple task learning. First, this work considers the concept of body schema and suggests a computational translation of this concept, appropriate for controlling a humanoid robot. This model of the body schema is adaptive and evolves as a result of the robot sensory experience. It suggests new avenues for understanding various psychophysical and neuropsychological phenomenons of human peripersonal space representation such as adaptation to distorted vision and tool use, fake limbs experiments, body-part centered receptive fields, and multimodal neurons. Second, it is shown how the motor modality can be added to the body schema. The suggested controller is inspired by the dynamical system theory of motor control and allows the robot to simultaneously and robustly control its limbs in joint angles space and in end-effector location space. This amounts to controlling the robot in both proprioceptive and visual modalities. This multimodal control can benefit from the advantages offered by each modality and is better than traditional robotic controllers in several respects. It offers a simple and elegant solution to the singularity and joint limit avoidance problems and can be seen as a generalization of the Damped Least Square approach to robot control. The controller exhibits several properties of human reaching movements, such as quasi-straight hand paths and bell-shaped velocity profiles and non-equifinality. In a third step, the motor modalities is endowed with a statistical learning mechanism, based on Gaussian Mixture Models, that enables the humanoid to learn motor primitives from demonstrations. The robot is thus able to learn simple manipulation tasks and generalize them to various context, in a way that is robust to perturbations occurring during task execution. In addition to simulation results, the whole model has been implemented and validated on two humanoid robots, the Hoap3 and the iCub, enabling them to learn their arm and head geometries, perform reaching movements, adapt to unknown tools, and visual distortions, and learn simple manipulation tasks in a smooth, robust and adaptive way. Finally, this work hints at possible computational interpretations of the concepts of body schema, motor perception and motor primitives.

EPFL2009