Influence of force and torque feedback on operator performance in a VR-based suturing task

The introduction of Minimally Invasive Surgery (MIS) has revolutionised surgical care, considerably improving the quality of many surgical procedures. Technological advances, particularly in robotic surgery systems, have reduced the complexity of such an approach, paving the way for even less invasive surgical trends. However, the fact that haptic feedback has been progressively lost through this transition is an issue that to date has not been solved. Whereas traditional open surgery provides full haptic feedback, the introduction of MIS has eliminated the possibility of direct palpation and tactile exploration. Nevertheless, these procedures still provide a certain amount of force feedback through the rigid laparoscopic tool. Many of the current telemanipulated robotic surgical systems in return do not provide full haptic feedback, which to a certain extent can be explained by the requirement of force sensors integrated into the tools of the slave robot and actuators in the surgeon's master console. In view of the increased complexity and cost, the benefit of haptic feedback is open to dispute. Nevertheless, studies have shown the importance of haptic feedback, especially when visual feedback is unreliable or absent. In order to explore the importance of haptic feedback for the surgeon's master console of a novel teleoperated robotic surgical system, we have identified a typical surgical task where performance could potentially be improved by haptic feedback, and investigate performance with and without this feedback. Two rounds of experiments are performed with 10 subjects, six of them with a medical background. Results show that feedback conditions, including force feedback, significantly improve task performance independently of the operator's suturing experience. There is, however, no further significant improvement when torque feedback is added. Consequently, it is deduced that force feedback in translations improves subject's dexterity, while torque feedback might not further benefit such a task.

Tactile Feedback Improves Performance in a Palpation Task: Results in a VR-Based Testbed

Hannes Bleuler, Roger Gassert, Kaspar Leuenberger, Evren Samur, Laura Santos Carreras

Robotic surgery provides many benefits such as reduced invasiveness and increased dexterity. This comes at the cost of no direct contact between surgeon and patient This physical separation prevents surgeons from performing direct haptic exploration of tissues and organs, imposing exclusive reliance on visual cues. Current technology is not yet able to both measure and reproduce a realistic and complete sense of touch (interaction force, temperature, roughness, etc.). In this paper, we put forward a concept based on multimodal feedback consisting of the integration of different kinds of visual and tactile cues with force feedback that can potentially improve both the surgeon's performance and the patient's safety. We present a cost-effective tactile display simulating a pulsating artery that has been integrated into a haptic work-station to combine both tactile and force-feedback information. Furthermore, we investigate the effect of different feedback types, including tactile and/or visual cues, on the performance of subjects carrying out two typical palpation tasks: (1) exploring a tissue to find a hidden artery and (2) identifying the orientation of a hidden artery. The results show that adding tactile feedback significantly reduces task completion time. Moreover, for high difficulty levels, subjects perform better with the feedback condition combining tactile and visual cues. As a matter of fact, the majority of the subjects in the study preferred this combined feedback because redundant feedback reassures subjects in their actions. Based on this work, we can infer that multimodal haptic feedback improves subjects' performance and confidence during exploratory procedures.

Mit Press2012

Foot-Interfaces Control and Human-Robot Interaction in Four-Handed Manipulation

Jacob Hernandez Sanchez

Humans have a great capacity to perform complex manipulations. However, there are many tasks for which two hands are not enough, such as in surgery. For such tasks, a dyad of people is required. However, when not used to work together, a dyad collaboration can be intermittent, slow and prone to misunderstandings if relied solely on verbal cues. The current PhD thesis report concerns the design and control of foot interfaces to enable solo four-handed manipulations. In the first part, we present the platform design in five DoF, with position mapping and haptic force feedback. We tackle the known issue of fatigue when using position mapping, by partially compensating for the dynamics of the leg. Finally, we perform a series of feasibility validations on bipedal teleoperation in 3 and 4 and 5 DoF. In the second part, a behavioural study investigates a symmetric bipedal holding and transport of an object while working on it with the hands. This study sheds light to the human capacity to use the redundancy given by augmentation with the feet, to assist the task of the hands. We investigate the effect of shared-control for lower limb teleoperation. Two modalities are used: 1. synergistic control of two robotic arms with one foot, 2. force assistance. We evaluate these strategies in terms of objective and subjective metrics, such as physical and cognitive workload and fluency.Aiming at enabling individualized foot dexterous control, in the third part of the thesis, we addressed the problem of bio-mechanical coupling of foot rotations. This is done by detecting intention to grasp, and selectively easing this action while haptically constraining the foot gestures for tool alignment. A user study was conducted to evaluate our proposed approach in a surgical laparoscopic gripper.In the fourth and final part, we focus on investigating more demanding tasks for the feet, as well as the coordinated control of four hands, during hybrid-robotic laparoscopic surgery. One foot controls a camera, while the other one performs a grasping task. Noteworthy is that all four-arms can be controlled simultaneously. For easing the solo surgical task, we design haptic assistance, for autonomous grasping operated by the foot. Finally, we perform a systematic behavioural study to understand the effect of our haptic-shared-control, along with an assistance modality for camera tool-tracking. We cover uni-pedal, bipedal and four-handed task scenarios. Our main conclusion is that foot gestures can be conditioned with haptic feedback, to effectively control multiple degrees of freedom of a supernumerary robotic arm. Thereby enabling four-arm simultaneous manipulations. This is done specifically by leveraging 1. a foot kinesthetic haptic platform that allows for the use of two feet, 2. leg compensation, and virtual assistive impedance, which are useful to increase comfort and alleviate fatigue. 3. Hybridization of control mappings for foot gestures, with limited range of motion and oblique axes of rotation, 4. Haptic shared-control strategies for prehensile force assistance, coordination assistance, and DoF-individuation assistance. We found that robotic assistance is highly appreciated, and has an important role in simplifying four-handed tasks. Indeed, these are instrumental considerations to enable a more fluent, less demanding, and more performant interaction, in tasks where the feet are supplementing the task of hands.

EPFL2022

Foot-Interfaces Control and Human-Robot Interaction in Four-Handed Manipulation

Jacob Hernandez Sanchez

EPFL2022