Haptic instrumentation for an interventional radiology simulator

Low trauma, reduced costs and fast recovery are only a few factors why minimally invasive surgery (MIS) is taking over classical surgical methods. Interventional Radiology (IR) is a minimally invasive procedure where thin tubular instruments are steered through the patient's vascular system under X-ray imaging. In order to perform these procedures, a radiologist has to be trained to master hand-eye coordination, instrument manipulation and procedure protocols. The importance of training IR procedures rises with the increasing variety and sophistication of the procedures performed using this approach. Apart from the traditional "see one, do one, teach one" model, radiologists have only had few mock devices with limitations in practical use. Training on animals is an alternative, but its use is hindered by high costs and ethical issues. The existing simulation systems all have major drawbacks: the requirement to use modified instruments, high inertia of the haptic feedback that creates a noticeably degraded dynamic behavior, or excessive friction that is not properly compensated. In this thesis we propose a quality training environment dedicated to IR. The system is composed of a virtual reality (VR) simulation of the patients anatomy linked to a robotic interface providing haptic force feedback. This thesis focuses on the requirements, design and prototyping of a specific haptic interface for the manipulation of catheters and guide wires. Translational tracking and force feedback on the catheter are provided by two cylinders forming a friction drive arrangement. The complete friction drive can be set in rotation with an additional motor providing torque feedback. A force and a torque sensor are integrated in the cylinders for direct measurement on the catheter enabling disturbance cancellation with a closed-loop force control strategy. One of the specific requirements during a simulated intervention is to measure the position of the tip of a guide wire inside a catheter. This is a so called "tube in tube" problem. The guide wire is detected at the entrance of the simulator and then grasped by a micro-gripper. Forces and movements of the guidewire moving inside the catheter are then transmitted through a gripper and along a rod to an external unit. This thesis also reports a study of several micro-gripper designs developed and tested on a haptic force feedback interface developed at the LSRO-EPFL. A virtual reality environment has been developed to provide real-time visualisation and force feedback. Several different software routines are integrated in the system. During an IR procedure the tip of an employed instrument interacts with a wall of blood vessels and a repulsive force is transmitted to the user. Force feedback helps radiologist to orient and navigate the inserted instrument in the vessel. This thesis presents a novel method for measurement of internal constraints during an IR procedure. The measurements help to determine the maximum interaction forces between the catheter and the wall of the blood vessel and to observe the shape of dynamic interaction during manipulation. In order to recreate a realistic simulation environment it is necessary to derive the elastic properties and behavior of arteries when contact with an instrument occurs. A novel method for measurement of mechanical properties of blood vessels is proposed in this work. Information derived with this method is also very useful to determine coherent boundary conditions for hemodynamic simulations. Measurements of the modulus of elasticity of an artery are carried out by measuring the deformations due to the inflation of an angioplasty balloon catheter used for IR procedures. A test bench, consisting of an angioplasty balloon, a PVA model and an actuator used to inflate the balloon, is developed for realization of experiments. The pressure-volume curve during the inflation of the balloon is observed. The Young's modulus is derived with an analytical model of the measurement system. The results are then analyzed and compared to existing data from literature.

Mechatronic elements and haptic rendering for computer-assisted minimally invasive surgery training

Thomas Moix

Technological advances of the last decades enabled the development of a set of new medical techniques that differ from traditional open surgery. These techniques, referred to as minimally invasive surgery (MIS), involve complex instruments and imaging devices to reach and treat anatomical regions through small incisions or natural entry points resulting in reduced trauma and shorter recovery time. For the surgeons, MIS introduces unnatural hand-eye coordination and anatomical representation that require specific training. One alternative is the use of virtual reality (VR) simulations of MIS procedures coupled to input devices mimicking instruments used by surgeons. To achieve high-fidelity with such a computer-assisted training system, the input device also provides the sensations experienced during a procedure with force reflective technology, often referred to as haptics. This work explores the mechatronic aspects implied in the realization of such a haptic device for MIS training applications. An analysis of different MIS procedures enables to draw two categories: lumen-guided and cavity procedures. A simple classification of the required mechanism can be extracted from these observations. To fulfil the requirements, an implementation should follow a series of guidelines in terms of actuation and power transmission that are presented. The interactions between a human operator and a haptic interface involve both force and motion. Impedance control generates a force command based on the position of the haptic interface. This process, called haptic rendering, is an integrate part of the VR environment. This work proposes to extend impedance control with model-based compensation of the dynamics and non-linearities (such as friction) of the haptic device. Due to sensitivity to model uncertainties, the proposed control architecture is completed by a parallel implicit force controller. Since the computational load of VR simulation for complex anatomies is high, update rates reach 20 to 30 Hz. These frequencies are not sufficient to control a haptic device and meet the sensitivity requirements of the human sensory system. Therefore, in such system a multirate approach is introduced. A model is proposed for the dynamics of a haptic interface and its user. Based on this model, the effect of the introduced implicit force controller is studied and different interfacing techniques between the VR simulation and the haptic controller are evaluated. Since force sensing is a requirement for the implementation of the proposed implicit forcecontroller, a technique using infrared reflective sensors is proposed. A modelling method for such sensors is introduced and implementation issues for force sensing are discussed. Finally, the topics discussed in this work are applied to the development of a computer-assisted training system for Interventional Radiology.

EPFL2005

Enactive robot vision

Mototaka Suzuki

The complexity of today's autonomous robots poses a major challenge for Artificial Intelligence. These robots are equipped with sophisticated sensors and mechanical abilities that allow them to enter our homes and interact with humans. For example, today's robots are almost all equipped with vision and several of them can move over rough terrain with wheels or legs. The methods developed so far in Artificial Intelligence, however, are not yet ready to cope with the complexity of the information gathered through the robot sensors and the need for rapid action in partially unknown and dynamic environments. In this thesis, I will argue that the apparent complexity of the environment and of the robot brain can be significantly simplified if perception, behavior, and learning are allowed to co-develop on the same time scale. In doing so, robots become sensitive to, and actively exploit, characteristics of the environment that they can tackle within their own computational and physical constraints. This line of work is grounded on philosophical and psychological research showing that perception is an active process mediated by behavior. However, computational models of active vision are very rare and often rely on architectures that are preprogrammed to detect certain characteristics of the environment. Previous work have shown that complex visual tasks, such as position and size invariant shape recognition as well as navigation, can be tackled with remarkably simple neural architectures generated by a coevolutionary process of active vision and feature selection. Behavioral machines equipped with primitive vision systems and direct pathways between visual and motor neurons were evolved while freely interacting with their environments. I proceed further on this line of investigation and describe the application of this methodology in three situations, namely car driving with an omnidirectional camera, goal-oriented navigation of a humanoid robot, and cooperative tasks by two agents. I will show that these systems develop sensitivity to a number of oriented, retinotopic, visual features – oriented edges, corners, height – and a behavioral repertoire to locate, bring, and keep these features in sensitive regions of the vision system that allow them to accomplish their goals. In a second set of experiments, I will show that active vision can be exploited by the robot to perform anticipatory exploration of the environment in a task that requires landmark-based navigation. Evolved robots exploit an internal expectation system that makes use of active exploration to check for expected events in the environment. I will describe a third set of experiments where, in addition to an evolutionary process, the visual system of the robot can develop receptive fields by means of unsupervised Hebbian learning and show that these receptive fields are significantly affected by the behavior of the system and differ from those predicted by most computational models of visual cortex. Finally, I will show that these robots replicate the performance deficiencies observed in experiments of motor deprivation with kitten when they are exposed to the same type of motor deprivations. Furthermore, the analyses of our robot brains suggest an explanation for the deficiencies observed in kitten that have not yet been fully understood.

EPFL2007

Wearable Technologies for Embodied Human-Robot Interaction

Carine Rognon

Robotic teleoperation is fundamental to augment the resilience, precision, and force of robots with the cognition of the operator. However, current interfaces, such as joysticks and remote controllers, are often complicated to handle since they require cognitive effort and learned skills. Wearable interfaces can enable more natural and intuitive interactions with robots, which would make robotic teleoperation accessible to a larger population of users for demanding tasks, such as manipulation or search-and-rescue. The aim of this thesis is to explore solutions to simplify our interactions with robots promoting their access to a broader range of the population. To achieve this, we are presenting a soft upper body exoskeleton, called the FlyJacket, for the bidirectional control of drones. Drones can greatly benefit us as they extend our perception and range of action. The exoskeleton controls a drone by recording torso movement and, through embedded haptic feedback devices, renders either kinesthetic guidance to improve the flight performance or tactile feedback to render the sensation of flying. We developed and tested an interface to control both a simulated and a real drone. The FlyJacket is a soft exoskeleton with arm support conceived to address the challenges of adapting to different morphologies and supporting the user during flight to prevent fatigue. We demonstrated that this novel interface allowed more consistent performance than when performing the same task with a remote controller and users felt more immersed into the flight. Interacting with a robot can be greatly enhanced by having multiple channels of sensorial feedback to increase the awareness of the operator. Information on the state of the drone can be intuitively rendered with haptic feedback. To create this bidirectional interaction with the drone, the two types of haptic feedback - kinesthetic and tactile - have been explored. Kinesthetic feedback was implemented with a cable-driven system to give guidance to the userâs torso position. Performing user studies, we could determine that the embedded guidance improved the flight performance and that a quadratically shaped force feedback curve was the most adequate profile to guide the user. We also established the minimal force difference, defined the perceived magnitude of this system and studied the learning process of users. Tactile feedback was investigated to render the sensation of flying by enhancing flight awareness, realism and immersion. To this end, we developed and embedded a new type of soft actuator that was compliant and lightweight such that it remained wearable and portable. Four devices, placed on the torso, provided feedback by compressing closed air pouches against the skin rendering the sensation of air pressure. A mechanical model and simulation of the pouch device were developed to determine appropriate parameters. We evaluated whether it conveyed useful information to the user and whether it enhanced the experience of flying. We demonstrated that users were able to understand the direction of the cues without prompting, could distinguish the cues quickly, and do so with high accuracy. The device was also used in a simulated flight task and users indicated that it increased the flight realism. We believe that the contributions of this thesis provides insights to the design of intuitive interfaces for human-robot interaction and increases their accessibility to a wider range of the population.

EPFL2019