Hand pose selection in a bimanual fine-manipulation task

Many daily tasks involve the collaboration of both hands. Humans dexterously adjust hand poses and modulate the forces exerted by fingers in response to task demands. Hand pose selection has been intensively studied in unimanual tasks, but little work has investigated bimanual tasks. This work examines hand poses selection in a bimanual high-precision-screwing task taken from watchmaking. Twenty right-handed subjects dismounted a screw on the watch face with a screwdriver in two conditions. Results showed that although subjects used similar hand poses across steps within the same experimental conditions, the hand poses differed significantly in the two conditions. In the free-base condition, subjects needed to stabilize the watch face on the table. The role distribution across hands was strongly influenced by hand dominance: the dominant hand manipulated the tool, whereas the nondominant hand controlled the additional degrees of freedom that might impair performance. In contrast, in the fixed-base condition, the watch face was stationary. Subjects used both hands even though single hand would have been sufficient. Importantly, hand poses decoupled the control of task-demanded force and torque across hands through virtual fingers that grouped multiple fingers into functional units. This preference for bimanual over unimanual control strategy could be an effort to reduce variability caused by mechanical couplings and to alleviate intrinsic sensorimotor processing burdens. To afford analysis of this variety of observations, a novel graphical matrix-based representation of the distribution of hand pose combinations was developed. Atypical hand poses that are not documented in extant hand taxonomies are also included.

Enhancing our understanding of human fine manipulation skills and advancing robot dexterity in grasping

Kunpeng Yao

From surgery to watchmaking, fine-manipulation skills highly rely on the dexterity afforded by both hands. Coordination is key to human dexterity. Specifically, humans need not only to govern the abundant intrinsic degrees of freedom (DOFs) to allocate controls of task-demanded variables, but also to adapt postures in response to extrinsic task conditions. In spite of the recent advances in robotics research, human dexterity remains unattainable for robots, especially in terms of flexibility and adaptability. Therefore, it is necessary to gain insights into human fine-manipulation skills, to advance the dexterity of robots in similar tasks. This thesis deepens our understanding of human dexterity by investigating the human coordination in fine-manipulation skills taken from watchmaking craftsmanship.The first part of this thesis investigates both intrinsic and extrinsic coordination of upper-limbs in a non-symmetrical and non-rhythmic bimanual fine-manipulation task. We conduct a comparative study in the assembly of a spring on the watch face. Analysis of motion kinematics reveals that professional subjects mitigate the task challenge by actively modifying task conditions. Moreover, we offer a novel perspective to understand the skill acquisition in humans. We hypothesize that the evolution of coordination during skill improvement is driven by the changes in the structure of the optimal criterion of the central nervous system. We employ a bi-level optimization framework to infer the structure of this optimal criterion.The second part investigates how roles and control variables are distributed across hands and fingers. We compare task performance of human subjects under two experimental conditions when dismounting a screw from a watch face. When the watch face needed positioning, the role distribution of both hands was strongly influenced by hand dominance; when the watch face was stationary, a variety of hand pose combinations emerged. We propose a taxonomy of bimanual hand pose combinations and develop a graphical matrix-based representation approach to afford analysis of experimental observations. Our analysis suggests that the control of independent task demands is distributed across either hands or functional groups of fingers.In the third part, we take inspiration from human coordination principle to advance the dexterity of a robotic hand by exploiting its redundant DOFs. We propose a human-like algorithm to enable a robotic hand to grasp objects using arbitrary surface regions, no longer restricted to regular grasp types, such as pinch or power grasp. We present an iterative process to empower the robotic hand to grasp multiple objects in sequence. Moreover, we formulate a strategy to facilitate the exploitation of redundant DOFs for multitask planning. Our approaches have been validated both in simulation and on a real robotic hand.In summary, this thesis not only offers a deeper understanding of human dexterity by revealing both the intrinsic and extrinsic coordination of upper-limbs and the role distribution across hands in bimanual fine-manipulation tasks, but also proposes algorithms that enable multi-fingered robotic hands to achieve human-like dexterous grasping of a single or even multiple objects using arbitrary surface regions. This thesis offers the prospect of developing algorithms for robotic dexterous grasping and manipulation, and also inspires the design of novel robotic hands and manipulators.

EPFL2022

Enhancing our understanding of human fine manipulation skills and advancing robot dexterity in grasping

Kunpeng Yao

EPFL2022