Human-Robot Interaction: Tackling the AI Challenges

Human-Robot interaction is an area full of challenges for artificial intelligence: dynamic, partially unknown environments that are not originally designed for autonomous machines; a large variety of situations and objects to deal with, with possibly complex semantics; physical interactions with humans that requires fine, low-latency control, representation and management of several mental models, pertinent situation assessment skills...the list goes on. This article sheds light on some key decisional issues that are to be tackled for a cognitive robot to share space and tasks with a human, and present our take on these challenges. We adopt a constructive approach based on the identification and the effective implementation of individual and collaborative skills. These cognitive abilities cover geometric reasoning and situation assessment mainly based on perspective-taking and affordances, management and exploitation of each agent (human and robot) knowledge in separate cognitive models, natural multi-modal communication, "human-aware" task planning, and human and robot interleaved plan achievement. We present our design choices, the articulations between the diverse deliberative components of the robot, experimental results, and eventually discuss the strengths and weaknesses of our approach. It appears that explicit knowledge management, both symbolic and geometric, proves to be key as it pushes for a different, more semantic way to address the decision-making issue in human-robot interactions.

A Few AI Challenges Raised while Developing an Architecture for Human-Robot Cooperative Task Achievement

Séverin Lemaignan

Over the last five years, and while developing an architecture for autonomous service robots in human environments, we have identified several key decisional issues that are to be tackled for a cognitive robot to share space and tasks with a human. We introduce some of them here: situation assessment and mutual modelling, management and exploitation of each agent (human and robot) knowledge in separate cognitive models, natural multi-modal communication, "human-aware" task planning, and human and robot interleaved plan achievement. As a general "take home" message, it appears that explicit knowledge management, both symbolic and geometric, proves to be a successful key while attempting to address these challenges, as it pushes for a different, more semantic way to address the decision-making issue in human-robot interactions.

2014

Cognitive Architecture for Mutual Modelling

Wafa Monia Benkaouar Johal, Pierre Dillenbourg, Alexis David Jacq, Ana Paiva

In social robotics, robots needs to be able to be understood by humans. Especially in collaborative tasks where they have to share mutual knowledge. For instance, in an educative scenario, learners share their knowledge and they must adapt their behaviour in order to make sure they are understood by others. Learners display behaviours in order to show their understanding and teachers adapt in order to make sure that the learners' knowledge is the required one. This ability requires a model of their own mental states perceived by others: ``has the human understood that I(robot) need this object for the task or should I explain it once again ?" In this paper, we discuss the importance of a cognitive architecture enabling second-order Mutual Modelling for Human-Robot Interaction in educative contexts

2016

Control of Contact Tasks in Autonomous and Human-Robot Collaborative Scenarios: A Dynamical System Approach

Walid Amanhoud

Robotic systems are more and more present in our daily life. Robots became indeed more economical, easy to program, safe and collaborative in their interaction with humans, versatile in performing different tasks and adaptive towards changes in the environment. Despite the recent advances in robotics, developingfully autonomous robots to achieve complex industrial or daily-life tasks is still very challenging. The human, due to his superior perception, understanding and reasoning skills, especially in uncertain environment, needs to be in the loop. Therefore, the ability of robots to collaborate with humans is of great interest inthe robotics community. Human-Robot Collaboration usually implies robots that share the control of the tasks with humans, providing adjustable autonomy level and mutual adaptation. A field that can particularly benefits from such collaboration is assistive robotics surgery. The introduction of shared autonomyin surgical tasks has several advantages. In particular, it can provide higher accuracy, flexibility, control and efficiency during the operation, over longer time periods, relieve the surgeon during repetitive tasks and reduce his mental workload. However, it is not very clear how to best use shared control to assist asurgeon during the operation. As part of a research project (the "Surgical project"), in this thesis we would like to propose solutions to that problem by building a teleoperated shared control architecture for a laparoscopic surgery scenario. While laparoscopic surgery usually involves one surgeon and two humanassistants to perform the surgery, the project aims to replace the assistants by two partially teleoperated robotic systems. The target shared control architecture would explore various strategies to ensure safety for both the patient and the surgeon, provide accuracy and stability in motion and force control of the toolsattached to the robots, reduce the mental workload of the surgeon and adapt the level of assistance based on well-identified criteria.The first year was mainly devoted to the control of motion and force for contact tasks which is of great importance in the context of the project. Robot force control is traditionally achieved either explicitly through hybrid position/force control or implicitly using impedance control. We propose an approach leveraging theproperties of dynamical systems (DS) for immediate re-planning and robustness to real-time perturbations by exploiting local modulation of the DS. Dynamical systems has been widely used as a tool to represent and generate motion in robotics application. Here the idea is to propose a DS formulation to generatecontact force in addition to motion. First implementation and experiments to evaluate the effectiveness of the strategy were conducted in practice with a 7-DOF robotic arm (KUKA LWR IV+).So far we focused on contact tasks that were performed autonomously by one or two robots. Future work will improve and use the proposed DS approach to consider contact tasks in shared control scenarios where one human and one or two robots need to collaborate to successfully achieve the tasks. Because of the nature of the surgical project we will use teleoperation systems to introduce the human in the loop. The developed teleoperated shared control architecture will be evaluated on laparoscopic surgery related tasks.

EPFL2021

Control of Contact Tasks in Autonomous and Human-Robot Collaborative Scenarios: A Dynamical System Approach

Walid Amanhoud

EPFL2021