Publications by Walid Amanhoud

Enabling four-arm laparoscopic surgery by controlling two robotic assistants via haptic foot interfaces

Walid Amanhoud, Aude Billard, Mohamed Bouri, Jacob Hernandez Sanchez

Robotic surgery is a promising direction to improve surgeons and assistants’ daily life with respect to conventional surgery. In this work, we propose solo laparoscopic surgery in which two robotic arms, controlled via haptic foot interfaces, assist the task of the hands. Such a system opens the door for simultaneous control of four laparoscopic tools by the same user. Each hand controls a manipulative tool while a foot controls an endoscope/camera and another controls an actuated gripper. In this scenario, the surgeon and robots need to work collaboratively within a concurrent workspace, while meeting the precision demands of surgery. To this end, we propose a control framework for the robotic arms that deals with all the task- and safety-related constraints. Furthermore, to ease the control through the feet, two assistance modalities are proposed: adaptive visual tracking of the laparoscopic instruments with the camera and grasping assistance for the gripper. A user study is conducted on twelve subjects to highlight the ease of use of the system and to evaluate the relevance of the proposed shared control strategies. The results confirm the feasibility of four-arm surgical-like tasks without extensive training in tasks that involve visual-tracking and manipulation goals for the feet, as well as coordination with both hands. Moreover, our study characterizes and motivates the use of robotic assistance for reducing task load, improving performance, increasing fluency, and eliciting higher coordination during four-arm laparoscopic tasks.

2023

Contact-Initiated Shared Control Strategies for Four-Arm Supernumerary Manipulation with Foot Interfaces

Walid Amanhoud, Aude Billard, Mohamed Bouri, Jacob Hernandez Sanchez

2021

Control of Contact Tasks in Autonomous and Human-Robot Collaborative Scenarios: A Dynamical System Approach

Walid Amanhoud

Robotic systems are more and more present in our daily life. Robots became indeed more economical, easy to program, safe and collaborative in their interaction with humans, versatile in performing different tasks and adaptive towards changes in the environment. Despite the recent advances in robotics, developingfully autonomous robots to achieve complex industrial or daily-life tasks is still very challenging. The human, due to his superior perception, understanding and reasoning skills, especially in uncertain environment, needs to be in the loop. Therefore, the ability of robots to collaborate with humans is of great interest inthe robotics community. Human-Robot Collaboration usually implies robots that share the control of the tasks with humans, providing adjustable autonomy level and mutual adaptation. A field that can particularly benefits from such collaboration is assistive robotics surgery. The introduction of shared autonomyin surgical tasks has several advantages. In particular, it can provide higher accuracy, flexibility, control and efficiency during the operation, over longer time periods, relieve the surgeon during repetitive tasks and reduce his mental workload. However, it is not very clear how to best use shared control to assist asurgeon during the operation. As part of a research project (the "Surgical project"), in this thesis we would like to propose solutions to that problem by building a teleoperated shared control architecture for a laparoscopic surgery scenario. While laparoscopic surgery usually involves one surgeon and two humanassistants to perform the surgery, the project aims to replace the assistants by two partially teleoperated robotic systems. The target shared control architecture would explore various strategies to ensure safety for both the patient and the surgeon, provide accuracy and stability in motion and force control of the toolsattached to the robots, reduce the mental workload of the surgeon and adapt the level of assistance based on well-identified criteria.The first year was mainly devoted to the control of motion and force for contact tasks which is of great importance in the context of the project. Robot force control is traditionally achieved either explicitly through hybrid position/force control or implicitly using impedance control. We propose an approach leveraging theproperties of dynamical systems (DS) for immediate re-planning and robustness to real-time perturbations by exploiting local modulation of the DS. Dynamical systems has been widely used as a tool to represent and generate motion in robotics application. Here the idea is to propose a DS formulation to generatecontact force in addition to motion. First implementation and experiments to evaluate the effectiveness of the strategy were conducted in practice with a 7-DOF robotic arm (KUKA LWR IV+).So far we focused on contact tasks that were performed autonomously by one or two robots. Future work will improve and use the proposed DS approach to consider contact tasks in shared control scenarios where one human and one or two robots need to collaborate to successfully achieve the tasks. Because of the nature of the surgical project we will use teleoperation systems to introduce the human in the loop. The developed teleoperated shared control architecture will be evaluated on laparoscopic surgery related tasks.

EPFL2021

Efficient Configuration Exploration in Inverse Dynamics Acquisition of Robotic Manipulators

Walid Amanhoud, Aude Billard, Sthithpragya Gupta, Farshad Khadivar

The inverse dynamics of a robotic manipulator is instrumental in precise robot control and manipulation. However, acquiring such a model is challenging, not only due to unmodelled non-linearities such as joint friction, but also from a machine learning perspective (e.g., input space dimension, amount of data needed). The accuracy of such models, regardless of the learning techniques, relies on proper excitation and exploration of the robot's configuration space, in order to collect a rich dataset. This study aims to provide rich data in learning the inverse dynamics of a serial robotic manipulator using supervised machine learning techniques. We propose a method, called Max-Information Configuration Exploration (MICE), to incrementally explore and generate information-rich data via computing parameters of a trajectory set. We also introduce a new set of excitation trajectories that explores robot's configuration through imposed stable limit cycles in robot joints' phase space while satisfying feasibility constraints and physical bounds. We benchmark MICE against state-of-the-art in terms of data quality and learning accuracy. The proposed methodology for data collection, model learning, and evaluation, is validated with a KUKA IIWA14 robotic arm where the results prove significant improvement over traditional approaches.

IEEE2021

Foot Control of a Surgical Laparoscopic Gripper via 5DoF Haptic Robotic Platform: Design, Dynamics and Haptic Shared Control

Walid Amanhoud, Aude Billard, Mohamed Bouri, Jacob Hernandez Sanchez

Foot devices have been ubiquitously used in surgery to control surgical equipment. Most common applications are foot switches for electro-surgery, endoscope positioning and tele-robotic consoles. Switches fall short of providing continuous control as required for precise use of instruments. We developed a haptic foot interface to provide continuous assistance in surgical procedures. This paper concerns the foot control of simultaneous five degrees of freedom (DoF) of a surgical laparoscopic gripper. We assess systematically precision at controlling position and orientation at the target and closing of the forceps. Our controller provides position:position mapping between the foot and the robotic tool, as well as haptic feedback, compensating for gravity of the lower limb of the operator so as to alleviate fatigue. A dynamic model compensation and closed loop force feedback is used to achieve high transparency and backdrivability. The assistance is based on a novel type of haptic fixtures combining spring-damper with selective dynamic compensation in the direction aligned with the task of grasping, so as to simplify control of certain poses, made difficult due to the coupling between human lower limbs’ DoF’s. We experimentally evaluated the control strategy with six users on a position control surgical task in simulation. Results show the proposed assistance greatly eases the foot grasping task leading to higher completeness, efficiency, and lower mental and physical load.

2021

Force Adaptation in Contact Tasks with Dynamical Systems

Walid Amanhoud, Aude Billard, Mahdi Khoramshahi

In many tasks such as finishing operations, achieving accurate force tracking is essential. However, uncertainties in the robot dynamics and the environment limit the force tracking accuracy. Learning a compensation model for these uncertainties to reduce the force error is an effective approach to overcome this limitation. However, this approach requires an adaptive and robust framework for motion and force generation. In this paper, we use the time-invariant Dynamical System (DS) framework for force adaptation in contact tasks. We propose to improve force tracking accuracy through online adaptation of a state-dependent force correction model encoded with Radial Basis Functions (RBFs). We evaluate our method with a KUKA LWR IV+ robotic arm. We show its efficiency to reduce the force error to a negligible amount with different target forces and robot velocities. Furthermore, we study the effect of the hyper-parameters and provide a guideline for their selection. We showcase a collaborative cleaning task with a human by integrating our method to previous works to achieve force, motion, and task adaptation at the same time. Thereby, we highlight the benefits of using adaptive force control in real-world environments where we need reactive and adaptive behaviours in response to interactions with the environment.

2020

A Dynamical System Approach to Motion and Force Generation in Contact Tasks

Walid Amanhoud, Aude Billard, Mahdi Khoramshahi

Many tasks require the robot to enter in contact with surfaces, be it to take support, to polish or to grasp an object. It is crucial that the robot controls forces both upon making contact and while in contact. While many solutions exist to control for contact, none offer the required robustness to adapt to real-world uncertainties, such as sudden displacement of the object prior and once in contact. To adapt to such disturbances require to re-plan on the fly both the trajectory and the force. Dynamical systems (DS) offer a framework for instant re-planning of trajectories. They are however limited to control of motions. We extend this framework here to enable generating contact forces and trajectories through DS. The framework allows also to modulate the impedance so as to show rigidity to maintain contact, and compliance to ensure safe interaction with humans. We validate the approach in single and dual arm setting using KUKA LWR 4+ robotic arms. We show that the approach allows 1) to make smooth contact while applying large forces, 2) to maintain desired contact force when scanning non-linear surfaces, even when the surface is moved, and 3) to grasp and lift smoothly an object in the air, and to re-balance forces on the fly to maintain the grasp even when subjected to strong external disturbances.

Robotics: Science and Systems (RSS)2019

A Dynamical System Approach to Motion and Force Generation in Contact Tasks

Walid Amanhoud, Aude Billard, Mahdi Khoramshahi

Many tasks require the robot to enter in contact with surfaces, be it to take support, to polish or to grasp an object. It is crucial that the robot controls forces both upon making contact and while in contact. While many solutions exist to control for contact, none offer the required robustness to adapt to real-world uncertainties, such as sudden displacement of the object prior and once in contact. To adapt to such disturbances require to re-plan on the fly both the trajectory and the force. Dynamical systems (DS) offer a framework for instant re-planning of trajectories. They are however limited to control of motions. We extend this framework here to enable generating contact forces and trajectories through DS. The framework allows also to modulate the impedance so as to show rigidity to maintain contact, and compliance to ensure safe interaction with humans. We validate the approach in single and dual arm setting using KUKA LWR 4+ robotic arms. We show that the approach allows 1) to make smooth contact while applying large forces, 2) to maintain desired contact force when scanning non-linear surfaces, even when the surface is moved, and 3) to grasp and lift smoothly an object in the air, and to re-balance forces on the fly to maintain the grasp even when subjected to strong external disturbances.

MIT PRESS2019

Four-Arm Manipulation via Feet Interfaces

Walid Amanhoud, Aude Billard, Hannes Bleuler, Mohamed Bouri, Anaïs Betsabeh Haget, Jacob Hernandez Sanchez

We seek to augment human manipulation by enabling humans to control two robotic arms in addition to their natural arms using their feet. Thereby, the hands are free to perform tasks of high dexterity, while the feet-controlled arms perform tasks requiring lower dexterity, such as supporting a load. The robotic arms are tele-operated through two foot interfaces that transmit translation and rotation to the end effector of the manipulator. Haptic feedback is provided for the human to perceive contact and change in load and to adapt the feet pressure accordingly. Existing foot interfaces have been used primarily for a single foot control and are limited in range of motion and number of degrees of freedom they can control. This paper presents foot-interfaces specifically made for bipedal control, with a workspace suitable for two feet operation and in five degrees of freedom each. This paper also presents a position-force teleoperation controller based on Impedance Control modulated through Dynamical Systems for trajectory generation. Finally, an initial validation of the platform is presented, whereby a user grasps an object with both feet and generates various disturbances while the object is supported by the feet.

AAAI2019