Towards Understanding of Human Kinematic Coordination Patterns in Bimanual Fine Manipulation Tasks

Skills that require the collaboration of both arms are common in humans’ everyday activities, and the coordination of upper limbs is key to human bimanual dexterity. The Skill Acquisition in Humans and Robots (ERC) project aims at obtaining a better understanding of the human skill acquisition process with a special focus on bimanual dexterous manipulation tasks, so as to develop novel robotic controllers resulting in the same level of precision and robustness. We conduct a longitudinal study of the acquisition of dexterous bimanual skills in watchmaking craftsmanship. We focus on the task of assembling a watch spring, where precise, coordinated, and directionally compliant control of the hands and fingers motions is required. We recorded the motion of subjects and the force/torque applied on the watch face during manipulation tasks. We compare the performance of 15 subjects participating in our experiments (10 novices and 5 experts). Our analysis shows that experts try to align the direction of force application demanded by the task with the optimal force application direction of the dominant arm. In addition, experts adjust finger placements to improve the dexterity of hands for manipulation. We assume the differences in performance across novices and experts can be explained by the differences in the weights of criteria (manipulability, task compatibility, and discomfort), which are used to evaluate the motion. We develop a computational model to infer the weights. Preliminary results show that novice subjects put higher weight on the task compatibility, while experts have higher weight on motion comfort.

Learning robotic eye–arm–hand coordination from human demonstration: a coupled dynamical systems approach

Aude Billard, Luka Lukic, José Santos-Victor

We investigate the role of obstacle avoidance in visually guided reaching and grasping movements. We report on a human study in which subjects performed prehensile motion with obstacle avoidance where the position of the obstacle was systematically varied across trials. These experiments suggest that reaching with obstacle avoidance is organized in a sequential manner, where the obstacle acts as an intermediary target. Furthermore, we demonstrate that the notion of workspace traveled by the hand is embedded explicitly in a forward planning scheme, which is actively involved in detecting obstacles on the way when performing reaching. We find that the gaze proactively coordinates the pattern of eye-arm motion during obstacle avoidance. This study provides also a quantitative assessment of the coupling between the eye-arm-hand motion. We show that the coupling follows regular phase dependencies, and is unaltered during obstacle avoidance. These observations provide a basis for the design of a computational model. Our controller extends the coupled dynamical systems (CDS) framework and provides fast and synchronous control of the eyes, the arm and the hand within a single and compact framework, mimicking similar control system found in humans. We validate our model for visuomotor control of a humanoid robot.

Springer2014

An inverse optimization approach to understand human acquisition of kinematic coordination in bimanual fine manipulation tasks

Aude Billard, Kunpeng Yao

Tasks that require the cooperation of both hands and arms are common in human everyday life. Coordination helps to synchronize in space and temporally motion of the upper limbs. In fine bimanual tasks, coordination enables also to achieve higher degrees of precision that could be obtained from a single hand. We studied the acquisition of bimanual fine manipulation skills in watchmaking tasks, which require assembly of pieces at millimeter scale. It demands years of training. We contrasted motion kinematics performed by novice apprentices to those of professionals. Fifteen subjects, ten novices and five experts, participated in the study. We recorded force applied on the watch face and kinematics of fingers and arms. Results indicate that expert subjects wisely place their fingers on the tools to achieve higher dexterity. Compared to novices, experts also tend to align task-demanded force application with the optimal force transmission direction of the dominant arm. To understand the cognitive processes underpinning the different coordination patterns across experts and novice subjects, we followed the optimal control theoretical framework and hypothesize that the difference in task performances is caused by changes in the central nervous system's optimal criteria. We formulated kinematic metrics to evaluate the coordination patterns and exploit inverse optimization approach to infer the optimal criteria. We interpret the human acquisition of novel coordination patterns as an alteration in the composition structure of the central nervous system's optimal criteria accompanied by the learning process.

2019

An inverse optimization approach to understand human acquisition of kinematic coordination in bimanual fine manipulation tasks

Aude Billard, Kunpeng Yao

2020