Social-Aware Coordination of Multi-robot Systems Based on Institutions

Institutional robotics (IR) is an approach to the coordination of multi-robot systems that draws inspiration from social sciences, namely from institutional economics. Using the concept of institution, it aims to provide a comprehensive strategy for specifying social interactions (e.g., norms, roles, hierarchies) among robots. In previous work, we have introduced a control methodology for multi-robot systems that takes into account institutions in order to create an Institutional Agent Controller (IAC) that captures such social interactions. In this chapter, the IAC design methodology is validated in a case study concerned with a swarm of 40 real, resource-constrained robots which has to maintain wireless connectivity. We then investigate a second case study dealing with more complex social interactions, showing that institutional roles can effectively help a multi-robot system to coordinate and improve performance in a given task of social nature. Given the fact that institutions are one of the tools in use within human societies to shape social interactions, our intuition is that IR can also facilitate coordination with humans in scenarios involving many-to-many human–robot interactions. We discuss how the IR concepts and the IAC design methodology can be implemented in real-world scenarios where multiple robots must interact with multiple humans in a socially aware manner.

From human-intention recognition to compliant control using dynamical systems in physical human-robot interaction

Mahdi Khoramshahi

Human ability to coordinate one's actions with other individuals to perform a task together is fascinating. For example, we coordinate our action with others when we carry a heavy object or when we construct a piece of furniture. Capabilities such as (1) force/compliance adaptation, (2) intention recognition, and (3) action/motion prediction enables us to assist others and fulfill the task. For instance, by adapting the compliance, we not only reject undesirable perturbations that undermine the task but also incorporate others' motions into the interaction. Complying with partners' motions allows us to recognize their intention and consequently predict their actions. With the growth of factories involving humans and robots working side by side, designing controllers and algorithms with such capacities is a crucial step toward assistive robotics. The challenge, however, is to attain a unified control strategy with predictive/adaptive capacities at the task, motion, and force-level which ensures a stable and safe interaction. To this aim, this thesis proposes a state-dependent dynamical system-based approach for prediction and control in physical human-robot interactions. In the first part of this dissertation, we focus on the human capacity to predict their partners' motion. More specifically, we investigate mechanisms of spatio-temporal coordination between two partners. We employ a simple scenario called âthe mirror gameâ where two individuals (human, robot, or avatar) imitate each other's motions. Our empirical assessment reveal that the intention-based prediction of the leader's motions allows the follower to compensate for perception-action delays and to improve the tracking performance in terms of temporal coordination and confidence. In the second part of this dissertation, we propose an adaptive mechanism that enables the robot to recognize the intention of a human user. We utilize state-dependent dynamical systems for motion planning and impedance control to deliver safe and compliant human-robot interaction. We consider a series of tasks (possible human intentions) encoded by dynamical system. Applying a similarity metric between the real velocities (induced by the human) and desired velocities generated by the dynamical systems, the robot is thus able to recognize the human's intention and switch to the intended task. We provide a rigorous experimental and analytical evaluation of our method yielding an interaction behavior that is safe and intuitive for the human. Finally, we tackle the compliance adaptation capability. We propose an admittance controller that reacts only when human-intentional forces are detected. Intentional and accidental forces are distinguished by measuring the persistency of the external forces, through a computation of the autocorrelation/energy of the force patterns. The overall controller exhibits variable stiffness where high stiffness allows the robot to reject the external disturbances and execute the task autonomously whereas low stiffness enables the robot to comply with human intentional forces. We demonstrate that our control architecture is effective in delivering satisfactory tracking and compliant behavior through a series of robotic experimentations.

EPFL2019

Learning How to Write with a Social Robot

Shruti Chandra

Learning handwriting is one of the main occupations for children in primary education. It is a complex skill that has been associated with children’s learning and development. Poor handwriting skills affect children negatively concerning their academic performance and motivation. In schools, children with handwriting problems need extra support and one-to-one interventions from teachers. Personalised one-to-one support requires additional resources that most schools generally cannot afford. Therefore, there is a need to explore different methods of learning and contemporary technologies to develop new educational tools to foster children’s learning process of handwriting. The research presented in this thesis addresses the question of - “How to create a social robot that can help children to acquire handwriting skills?". Thus, we looked into three different research domains: education; pedagogic sciences; and social human-robot interaction (HRI). On the one hand, we investigated the types of errors commonly present in children’s handwriting. On the other hand, we needed to explore different educational theories to find alternate learning methods that would maximise children’s learning gains. In addition, we investigated various possibilities for integrating a social robot into educational scenarios and measured its impact on the children’s learning process. This thesis presents five studies that follow an incremental evolution of the research. The first three studies were based on the interaction between two children, in the presence of either a human or a robotic facilitator, with the children performing a collaborative writing activity. We explored the ‘peer-learning’ and ‘peer-tutoring’ methods and compared the differences in terms of interaction behaviour and learning gains. Further, we analysed children’s interpersonal distance and feelings of responsibility towards their peers, both in the presence of a robot and human facilitator. Finally, we examined children’s assessment behaviour towards their peers and self-disclosure towards the robotic facilitator. The outcomes of these first studies provided enough data to generate a taxonomy of children handwriting errors to be used in the subsequent studies. They hinted that the ‘peer-tutoring’ method had a positive influence on children’s learning and corrective feedback to their peers in the presence of the social robot, providing positive support to the applicability of social robots as ‘facilitators’ in handwriting learning. The next step was to study the effects of the robot in the peer-tutoring scenario when assuming the role of a peer, directly interacting and performing a collaborative writing activity with the children. Thus, we developed an educational system with an autonomous social behaviour of a robot exhibiting various levels of learning competencies. The tutor-child provides corrective feedback on the writing of the learner-robot and in turn, enhances his writing skills. The robot’s writing was generated by an algorithm incorporating human-inspired movements and could reproduce a set of writing errors from the error taxonomy. We tested the system by conducting two longitudinal studies, each of them composed of four sessions. In the fourth study, we assigned the robot two contrasting competencies: ‘learning’ and ‘non-learning’. We measured the differences in children’s learning gains and changes in their perceptions towards the learner-robot. The fifth study followed a similar interaction scenario and research questions, but this time the robot performed three learning competencies: ‘continuous-learning’;’non-learning’ and ‘personalised-learning’. The findings of these studies show that the children improved more with the robot exhibiting learning, continuous-learning and personalised-learning competencies compared to the robot showing non-learning competency. The way children’s learning and perception of the robot evolved over time confirmed the need for multiple sessions for the last two studies. This thesis presents some evidence that social robots can be successfully used as tools for enhancing children’s handwriting skills in peer-learning scenarios.

EPFL2019

Control of Contact Tasks in Autonomous and Human-Robot Collaborative Scenarios: A Dynamical System Approach

Walid Amanhoud

Robotic systems are more and more present in our daily life. Robots became indeed more economical, easy to program, safe and collaborative in their interaction with humans, versatile in performing different tasks and adaptive towards changes in the environment. Despite the recent advances in robotics, developingfully autonomous robots to achieve complex industrial or daily-life tasks is still very challenging. The human, due to his superior perception, understanding and reasoning skills, especially in uncertain environment, needs to be in the loop. Therefore, the ability of robots to collaborate with humans is of great interest inthe robotics community. Human-Robot Collaboration usually implies robots that share the control of the tasks with humans, providing adjustable autonomy level and mutual adaptation. A field that can particularly benefits from such collaboration is assistive robotics surgery. The introduction of shared autonomyin surgical tasks has several advantages. In particular, it can provide higher accuracy, flexibility, control and efficiency during the operation, over longer time periods, relieve the surgeon during repetitive tasks and reduce his mental workload. However, it is not very clear how to best use shared control to assist asurgeon during the operation. As part of a research project (the "Surgical project"), in this thesis we would like to propose solutions to that problem by building a teleoperated shared control architecture for a laparoscopic surgery scenario. While laparoscopic surgery usually involves one surgeon and two humanassistants to perform the surgery, the project aims to replace the assistants by two partially teleoperated robotic systems. The target shared control architecture would explore various strategies to ensure safety for both the patient and the surgeon, provide accuracy and stability in motion and force control of the toolsattached to the robots, reduce the mental workload of the surgeon and adapt the level of assistance based on well-identified criteria.The first year was mainly devoted to the control of motion and force for contact tasks which is of great importance in the context of the project. Robot force control is traditionally achieved either explicitly through hybrid position/force control or implicitly using impedance control. We propose an approach leveraging theproperties of dynamical systems (DS) for immediate re-planning and robustness to real-time perturbations by exploiting local modulation of the DS. Dynamical systems has been widely used as a tool to represent and generate motion in robotics application. Here the idea is to propose a DS formulation to generatecontact force in addition to motion. First implementation and experiments to evaluate the effectiveness of the strategy were conducted in practice with a 7-DOF robotic arm (KUKA LWR IV+).So far we focused on contact tasks that were performed autonomously by one or two robots. Future work will improve and use the proposed DS approach to consider contact tasks in shared control scenarios where one human and one or two robots need to collaborate to successfully achieve the tasks. Because of the nature of the surgical project we will use teleoperation systems to introduce the human in the loop. The developed teleoperated shared control architecture will be evaluated on laparoscopic surgery related tasks.

EPFL2021

Learning How to Write with a Social Robot

Shruti Chandra

EPFL2019

From human-intention recognition to compliant control using dynamical systems in physical human-robot interaction

Mahdi Khoramshahi

EPFL2019

Control of Contact Tasks in Autonomous and Human-Robot Collaborative Scenarios: A Dynamical System Approach

Walid Amanhoud

EPFL2021