Biologically Inspired Kinematic Synergies Provide a New Paradigm for Balance Control of Humanoid Robots

Nature has developed methods for controlling the movements of organisms with many degrees of freedom which differ strongly from existing approaches for balance control in humanoid robots: Biological organisms employ kinematic synergies that simultaneously engage many joints, and which are apparently designed in such a way that their superposition is approximately linear. We show in this article that this control strategy can in principle also be applied to balance control of humanoid robots. In contrast to existing approaches, this control strategy reduces the need to carry out complex computations in real time (replacing the iterated solution of quadratic optimization problems by a simple linear controller), and it does not require knowledge of a dynamic model of the robot. Therefore it can handle unforeseen changes in the dynamics of the robot that may for example arise from wind or other external forces. We demonstrate the feasibility of this novel approach to humanoid balance control through simulations of the humanoid robot HOAP-2 for tasks that require balance control on a randomly moving surfboard.

Motion capture and reinforcement learning of dynamically stable humanoid movement primitives

Tadej Petric

Direct transfer of human motion trajectories to humanoid robots does not result in dynamically stable robot movements due to the differences in human and humanoid robot kinematics and dynamics. We developed a system that converts human movements captured by a low-cost RGB-D camera into dynamically stable humanoid movements. The transfer of human movements occurs in real-time. As need arises, the developed system can smoothly transition between unconstrained movement imitation and imitation with balance control, where movement reproduction occurs in the null space of the balance controller. The developed balance controller is based on an approximate model of the robot dynamics, which is sufficient to stabilize the robot during on-line imitation. However, the resulting movements cannot be guaranteed to be optimal because the model of the robot dynamics is not exact. The initially acquired movement is therefore subsequently improved by model-free reinforcement learning, both with respect to the accuracy of reproduction and balance control. We present experimental results in simulation and on a real humanoid robot.

IEEE2013

Visuomotor Coordination in Reach-To-Grasp Tasks: From Humans to Humanoids and Vice Versa

Luka Lukic

Understanding the principles involved in visually-based coordinated motor control is one of the most fundamental and most intriguing research problems across a number of areas, including psychology, neuroscience, computer vision and robotics. Not very much is known regarding computational functions that the central nervous system performs in order to provide a set of requirements for visually-driven reaching and grasping. Additionally, in spite of several decades of advances in the field, the abilities of humanoids to perform similar tasks are by far modest when needed to operate in unstructured and dynamically changing environments. More specifically, our first focus is understanding the principles involved in human visuomotor coordination. Not many behavioral studies considered visuomotor coordination in natural, unrestricted, head-free movements in complex scenarios such as obstacle avoidance. To fill this gap, we provide an assessment of visuomotor coordination when humans perform prehensile tasks with obstacle avoidance, an issue that has received far less attention. Namely, we quantify the relationships between the gaze and arm-hand systems, so as to inform robotic models, and we investigate how the presence of an obstacle modulates this pattern of correlations. Second, to complement these observations, we provide a robotic model of visuomotor coordination, with and without the presence of obstacles in the workspace. The parameters of the controller are solely estimated by using the human motion capture data from our human study. This controller has a number of interesting properties. It provides an efficient way to control the gaze, arm and hand movements in a stable and coordinated manner. When facing perturbations while reaching and grasping, our controller adapts its behavior almost instantly, while preserving coordination between the gaze, arm, and hand. In the third part of the thesis, we study the neuroscientific literature of the primates. We here stress the view that the cerebellum uses the cortical reference frame representation. The cerebellum by taking into account this representation performs closed-loop programming of multi-joint movements and movement synchronization between the eye-head system, arm and hand. Based on this investigation, we propose a functional architecture of the cerebellar-cortical involvement. We derive a number of improvements of our visuomotor controller for obstacle-free reaching and grasping. Because this model is devised by carefully taking into account the neuroscientific evidence, we are able to provide a number of testable predictions about the functions of the central nervous system in visuomotor coordination. Finally, we tackle the flow of the visuomotor coordination in the direction from the arm-hand system to the visual system. We develop two models of motor-primed attention for humanoid robots. Motor-priming of attention is a mechanism that implements prioritizing of visual processing with respect to motor-relevant parts of the visual field. Recent studies in humans and monkeys have shown that visual attention supporting natural behavior is not exclusively defined in terms of visual saliency in color or texture cues, rather the reachable space and motor plans present the predominant source of this attentional modulation. Here, we show that motor-priming of visual attention can be used to efficiently distribute robot's computational resources devoted to visual processing.

EPFL2015

Tales from a Robotic World. How Intelligent Machines Will Shape Our Future

Dario Floreano, Nicola Nosengo

Stories from the future of intelligent machines—from rescue drones to robot spouses—and accounts of cutting-edge research that could make it all possible. Tech prognosticators promised us robots—autonomous humanoids that could carry out any number of tasks. Instead, we have robot vacuum cleaners. But, as Dario Floreano and Nicola Nosengo report, advances in robotics could bring those rosy predictions closer to reality. A new generation of robots, directly inspired by the intelligence and bodies of living organisms, will be able not only to process data but to interact physically with humans and the environment. In this book, Floreano, a roboticist, and Nosengo, a science writer, bring us tales from the future of intelligent machines—from rescue drones to robot spouses—along with accounts of the cutting-edge research that could make it all possible. These stories from the not-so-distant future show us robots that can be used for mitigating effects of climate change, providing healthcare, working with humans on the factory floor, and more. Floreano and Nosengo tell us how an application of swarm robotics could protect Venice from flooding, how drones could reduce traffic on the congested streets of mega-cities like Hong Kong, and how a “long-term relationship model” robot could supply sex, love, and companionship. After each fictional scenario, they explain the technologies that underlie it, describing advances in such areas as soft robotics, swarm robotics, aerial and mobile robotics, humanoid robots, wearable robots, and even biohybrid robots based on living cells. Robotics technology is no silver bullet for all the world's problems—but it can help us tackle some of the most pressing challenges we face.

MIT Press2022

Visuomotor Coordination in Reach-To-Grasp Tasks: From Humans to Humanoids and Vice Versa

Luka Lukic

EPFL2015