Soft haptic device to render the sensation of flying like a drone

Haptic feedback located on the torso is proposed to enhance the state awareness of a user in virtual reality or during teleoperation while leaving the hands free for manipulation and communication. We provide haptic feedback on the torso by compressing a set of closed air pouches against the skin in order to render the sensation of air pressure when piloting a drone. The pouch devices are cable-driven and integrated in a wearable soft exoskeleton, called the FlyJacket. A mechanical model and simulation of a pouch device were developed in order to determine appropriate parameters, including the air pouch inner pressure, its attachment point, and the cable position. Using the simulation results, a set of pouch devices were constructed and integrated into the soft exoskeleton on both sides of the upper chest and middle of the back. The mechanical performance of the constructed device is close to that predicted by the simulation. Application of the haptic device in a flight task in which the user controls a drone using upper body movements was demonstrated with a user study. Adding haptic feedback during a stabilization task reduced the user's workload and improved the state awareness of the user.

Wearable Technologies for Embodied Human-Robot Interaction

Carine Rognon

Robotic teleoperation is fundamental to augment the resilience, precision, and force of robots with the cognition of the operator. However, current interfaces, such as joysticks and remote controllers, are often complicated to handle since they require cognitive effort and learned skills. Wearable interfaces can enable more natural and intuitive interactions with robots, which would make robotic teleoperation accessible to a larger population of users for demanding tasks, such as manipulation or search-and-rescue. The aim of this thesis is to explore solutions to simplify our interactions with robots promoting their access to a broader range of the population. To achieve this, we are presenting a soft upper body exoskeleton, called the FlyJacket, for the bidirectional control of drones. Drones can greatly benefit us as they extend our perception and range of action. The exoskeleton controls a drone by recording torso movement and, through embedded haptic feedback devices, renders either kinesthetic guidance to improve the flight performance or tactile feedback to render the sensation of flying. We developed and tested an interface to control both a simulated and a real drone. The FlyJacket is a soft exoskeleton with arm support conceived to address the challenges of adapting to different morphologies and supporting the user during flight to prevent fatigue. We demonstrated that this novel interface allowed more consistent performance than when performing the same task with a remote controller and users felt more immersed into the flight. Interacting with a robot can be greatly enhanced by having multiple channels of sensorial feedback to increase the awareness of the operator. Information on the state of the drone can be intuitively rendered with haptic feedback. To create this bidirectional interaction with the drone, the two types of haptic feedback - kinesthetic and tactile - have been explored. Kinesthetic feedback was implemented with a cable-driven system to give guidance to the userâs torso position. Performing user studies, we could determine that the embedded guidance improved the flight performance and that a quadratically shaped force feedback curve was the most adequate profile to guide the user. We also established the minimal force difference, defined the perceived magnitude of this system and studied the learning process of users. Tactile feedback was investigated to render the sensation of flying by enhancing flight awareness, realism and immersion. To this end, we developed and embedded a new type of soft actuator that was compliant and lightweight such that it remained wearable and portable. Four devices, placed on the torso, provided feedback by compressing closed air pouches against the skin rendering the sensation of air pressure. A mechanical model and simulation of the pouch device were developed to determine appropriate parameters. We evaluated whether it conveyed useful information to the user and whether it enhanced the experience of flying. We demonstrated that users were able to understand the direction of the cues without prompting, could distinguish the cues quickly, and do so with high accuracy. The device was also used in a simulated flight task and users indicated that it increased the flight realism. We believe that the contributions of this thesis provides insights to the design of intuitive interfaces for human-robot interaction and increases their accessibility to a wider range of the population.

EPFL2019

Haptic instrumentation for an interventional radiology simulator

Dejan Ilic

Low trauma, reduced costs and fast recovery are only a few factors why minimally invasive surgery (MIS) is taking over classical surgical methods. Interventional Radiology (IR) is a minimally invasive procedure where thin tubular instruments are steered through the patient's vascular system under X-ray imaging. In order to perform these procedures, a radiologist has to be trained to master hand-eye coordination, instrument manipulation and procedure protocols. The importance of training IR procedures rises with the increasing variety and sophistication of the procedures performed using this approach. Apart from the traditional "see one, do one, teach one" model, radiologists have only had few mock devices with limitations in practical use. Training on animals is an alternative, but its use is hindered by high costs and ethical issues. The existing simulation systems all have major drawbacks: the requirement to use modified instruments, high inertia of the haptic feedback that creates a noticeably degraded dynamic behavior, or excessive friction that is not properly compensated. In this thesis we propose a quality training environment dedicated to IR. The system is composed of a virtual reality (VR) simulation of the patients anatomy linked to a robotic interface providing haptic force feedback. This thesis focuses on the requirements, design and prototyping of a specific haptic interface for the manipulation of catheters and guide wires. Translational tracking and force feedback on the catheter are provided by two cylinders forming a friction drive arrangement. The complete friction drive can be set in rotation with an additional motor providing torque feedback. A force and a torque sensor are integrated in the cylinders for direct measurement on the catheter enabling disturbance cancellation with a closed-loop force control strategy. One of the specific requirements during a simulated intervention is to measure the position of the tip of a guide wire inside a catheter. This is a so called "tube in tube" problem. The guide wire is detected at the entrance of the simulator and then grasped by a micro-gripper. Forces and movements of the guidewire moving inside the catheter are then transmitted through a gripper and along a rod to an external unit. This thesis also reports a study of several micro-gripper designs developed and tested on a haptic force feedback interface developed at the LSRO-EPFL. A virtual reality environment has been developed to provide real-time visualisation and force feedback. Several different software routines are integrated in the system. During an IR procedure the tip of an employed instrument interacts with a wall of blood vessels and a repulsive force is transmitted to the user. Force feedback helps radiologist to orient and navigate the inserted instrument in the vessel. This thesis presents a novel method for measurement of internal constraints during an IR procedure. The measurements help to determine the maximum interaction forces between the catheter and the wall of the blood vessel and to observe the shape of dynamic interaction during manipulation. In order to recreate a realistic simulation environment it is necessary to derive the elastic properties and behavior of arteries when contact with an instrument occurs. A novel method for measurement of mechanical properties of blood vessels is proposed in this work. Information derived with this method is also very useful to determine coherent boundary conditions for hemodynamic simulations. Measurements of the modulus of elasticity of an artery are carried out by measuring the deformations due to the inflation of an angioplasty balloon catheter used for IR procedures. A test bench, consisting of an angioplasty balloon, a PVA model and an actuator used to inflate the balloon, is developed for realization of experiments. The pressure-volume curve during the inflation of the balloon is observed. The Young's modulus is derived with an analytical model of the measurement system. The results are then analyzed and compared to existing data from literature.

EPFL2005

Haptic interface design and control with application to surgery simulation

Ulrich Spälter

This thesis proposes a framework for haptic devices interacting with virtual environments. As application of the theoretical results a haptic device for the surgical training of hysteroscopy, a gynecologic intervention, is presented. Surgery simulators, which can be described as 'flight simulators for surgeons', comprise a virtual reality for modeling and visualization of organs, and the haptic device: While the surgeon interactively follows the scene on the computer screen, he perceives the contact forces at the tool handle of the surgical instrument. The haptic device tracks the position of the surgery tool and transforms the virtual contact forces to real forces, which can be actually felt by the surgeon. This thesis treats haptic devices with application to surgery simulators for minimally-invasive surgery. Haptic interfaces are complex mechatronic systems and show inherent trade-offs between haptic realism and system stability. Human factors play an important role, as the human operator can both stabilize or destabilize the system. The haptic perception, which can vary between individuals, as a criterion for haptic rendering quality is still not formalized. The optimization of haptic interfaces, within focus of research since 15 years ago, has remained a complex task. However, with look at the emerging applications for haptics, it has become an even more important topic. For this reason it is necessary to integrate mechanical design, control design, mechatronic elements and human factors in a unifying approach. This thesis highlights the principal trade-offs of haptic interfaces and works out design guidelines for new developments and quantitative information on where and how to optimize haptic systems. As application example a haptic interface for the simulation of hysteroscopy, a gynecologic intervention on the female uterus, has been realized as prototype. The device takes the characteristics of hysteroscopy into account and introduces features like insertion and complete removal of the surgery tool from the haptic interface. A design methodology is proposed, which combines aspects of theoretical kinematics based on Lie-Groups with integrated product development – a systematic approach for engineering tasks. The result, a novel kinematic design, has been realized as haptic interface prototype. The well-known affine parameterization for all stabilizing controllers is shown to be a well-suited tool for haptic control synthesis. It provides insights into the control trade-offs and allows to integrate human factors at controller design stage. The resulting controller, demonstrated by experimental results, can reduce parasitic effects (friction, device dynamics) close to and below the human perception threshhold, thus making the device transparent within the specified bandwidth and suitable for real-time implementation. For numerical controller representation the δ-operator is discussed. It unifies continuous and discrete time domain and has beneficial numerical properties. As there is still no standardized procedure for technical performance evaluation of haptic interfaces methods proposed in literature are summarized. In the near future a large market for haptic technology can be expected with applications from surgery-assist devices and drive-by-wire interfaces in automobiles to entertainment, which highlights the relevance of the treated topic.

EPFL2006