Arm-wrist haptic sleeve for drone teleoperation

Teleoperators rely on both visual and haptic feedback to perform drone teleoperation tasks, such as obstacle avoidance. Haptic feedback becomes essential when visual feedback is compromised, either due to visual occlusions or poor depth perception. However, haptic interfaces, are often bulky because they require heavy actuators to provide force feedback. The bulkiness reduces user mobility and makes these interfaces unsuitable for prolonged use. Here, we propose a wearable haptic sleeve that encompasses the wrist and elbow joints of a human arm. The two joint rotations control the motion of a drone in a simulated environment along a horizontal plane. The sleeve is composed of modular electroadhesive clutches that block the joint movement when the drone is in the vicinity of an obstacle. The clutches are lightweight (27 g), require low power (~1 mW) to operate, and can be mounted on the user without affecting the user's mobility. A motor learning subject study is conducted to navigate a drone through a hole in a wall where the depth perception of visual feedback is compromised. The results of the study show that subjects trained with the wearable haptic sleeve learnt the drone obstacle avoidance task and retained the necessary motor skills after haptic training, compared to subjects who received only visual feedback and were unable to learn the motor task.

Wearable Technologies for Embodied Human-Robot Interaction

Carine Rognon

Robotic teleoperation is fundamental to augment the resilience, precision, and force of robots with the cognition of the operator. However, current interfaces, such as joysticks and remote controllers, are often complicated to handle since they require cognitive effort and learned skills. Wearable interfaces can enable more natural and intuitive interactions with robots, which would make robotic teleoperation accessible to a larger population of users for demanding tasks, such as manipulation or search-and-rescue. The aim of this thesis is to explore solutions to simplify our interactions with robots promoting their access to a broader range of the population. To achieve this, we are presenting a soft upper body exoskeleton, called the FlyJacket, for the bidirectional control of drones. Drones can greatly benefit us as they extend our perception and range of action. The exoskeleton controls a drone by recording torso movement and, through embedded haptic feedback devices, renders either kinesthetic guidance to improve the flight performance or tactile feedback to render the sensation of flying. We developed and tested an interface to control both a simulated and a real drone. The FlyJacket is a soft exoskeleton with arm support conceived to address the challenges of adapting to different morphologies and supporting the user during flight to prevent fatigue. We demonstrated that this novel interface allowed more consistent performance than when performing the same task with a remote controller and users felt more immersed into the flight. Interacting with a robot can be greatly enhanced by having multiple channels of sensorial feedback to increase the awareness of the operator. Information on the state of the drone can be intuitively rendered with haptic feedback. To create this bidirectional interaction with the drone, the two types of haptic feedback - kinesthetic and tactile - have been explored. Kinesthetic feedback was implemented with a cable-driven system to give guidance to the userâs torso position. Performing user studies, we could determine that the embedded guidance improved the flight performance and that a quadratically shaped force feedback curve was the most adequate profile to guide the user. We also established the minimal force difference, defined the perceived magnitude of this system and studied the learning process of users. Tactile feedback was investigated to render the sensation of flying by enhancing flight awareness, realism and immersion. To this end, we developed and embedded a new type of soft actuator that was compliant and lightweight such that it remained wearable and portable. Four devices, placed on the torso, provided feedback by compressing closed air pouches against the skin rendering the sensation of air pressure. A mechanical model and simulation of the pouch device were developed to determine appropriate parameters. We evaluated whether it conveyed useful information to the user and whether it enhanced the experience of flying. We demonstrated that users were able to understand the direction of the cues without prompting, could distinguish the cues quickly, and do so with high accuracy. The device was also used in a simulated flight task and users indicated that it increased the flight realism. We believe that the contributions of this thesis provides insights to the design of intuitive interfaces for human-robot interaction and increases their accessibility to a wider range of the population.

EPFL2019

Personalized Body-Machine Interfaces for Advanced Human-Robot Interaction

Matteo Macchini

Robotic systems are becoming more and more pervasive in modern industrial, scientific and personal activities through recent years and will play a fundamental role in the future society. Despite their increasing level of automation, teleoperation is still needed in many robotic applications. Human-Robot Interfaces (HRIs) are a central element in the diffusion of robots and are a crucial component to make them available to the use of most people. However, standard interfaces such as remote controllers still need time and extensive training to be proficiently mastered by users. Body-Machine Interfaces (BoMIs) represent a promising alternative to such devices as they leverage the innate ability of humans to finely control their movements. This thesis aims to provide new insights about humans' natural motor behaviors when interfacing with robots and to propose novel approaches for the implementation of HRIs based on body motion. The core contribution of this work is the design, realization and qualification of machine learning-based methods to translate the motion of a person into control inputs for a robot. The methods described here allowed naive users to effectively control real and simulated robots based on the processing of their body movements with a faster learning rate compared to classic interfaces. We extended our core algorithm to leverage knowledge about the natural organization of human motion resulting a more general method that was successfully applied for the control of a set of morphologically different machines. Through the implementation of an online adaptive functionality, we enabled users to modify their motor behaviour during teleoperation resulting in a lower perceived physical and cognitive workload and augmented performance. Human factors such as the sense of presence in virtual or distant environments, or the motor characteristics of distinct body areas are a key factor for advanced telerobotic applications. We studied the impact of the use of Virtual Reality for the control of non-anthropomorphic machines, and the effects of designing BoMIs based on different limbs opening to new knowledge about the effects of such factors in human-robot interaction. The design of a wearable haptic interface for drone operation showed the potential of augmenting their senses to perceive a distal environment through the sense of touch. Thanks to this novel design, nonexpert users were enabled to navigate drones through narrow passages with limited visibility, a task almost impossible to perform with standard interfaces.We believe that this thesis contributes to pave the way towards a deeper understanding of humans in human-robot interaction, as well as providing a new set of tools for the design of simple, intuitive interfaces with the goal of democratizing robotic teleoperation for the masses.

EPFL2021

Personalized Body-Machine Interfaces for Advanced Human-Robot Interaction

Matteo Macchini

EPFL2021

Wearable Technologies for Embodied Human-Robot Interaction

Carine Rognon

EPFL2019