When Positive Perception of the Robot Has No Effect on Learning

Humanoid robots, with a focus on personalised social behaviours, are increasingly being deployed in educational settings to support learning. However, crafting pedagogical HRI designs and robot interventions that have a real, positive impact on participants' learning, as well as effectively measuring such impact, is still an open challenge. As a first effort in tackling the issue, in this paper we propose a novel robot-mediated, collaborative problem solving activity for school-children, called JUSThink, aiming at improving their computational thinking skills. JUSThink will serve as a baseline and reference for investigating how the robot's behaviour can influence the engagement of the children with the activity, as well as their collaboration and mutual understanding while working on it. To this end, this first iteration aims at investigating (i) participants' engagement with the activity (Intrinsic Motivation Inventory-IMI), their mutual understanding (IMI-like) and perception of the robot (Godspeed Questionnaire); (ii) participants' performance during the activity, using several performance and learning metrics. We carried out an extensive user-study in two international schools in Switzerland, in which around 100 children participated in pairs in one-hour long interactions with the activity. Surprisingly, we observe that while a teams' performance significantly affects how team members evaluate their competence, mutual understanding and task engagement, it does not affect their perception of the robot and its helpfulness, a fact which highlights the need for baseline studies and multi-dimensional evaluation metrics when assessing the impact of robots in educational activities.

Learning How to Write with a Social Robot

Shruti Chandra

Learning handwriting is one of the main occupations for children in primary education. It is a complex skill that has been associated with children’s learning and development. Poor handwriting skills affect children negatively concerning their academic performance and motivation. In schools, children with handwriting problems need extra support and one-to-one interventions from teachers. Personalised one-to-one support requires additional resources that most schools generally cannot afford. Therefore, there is a need to explore different methods of learning and contemporary technologies to develop new educational tools to foster children’s learning process of handwriting. The research presented in this thesis addresses the question of - “How to create a social robot that can help children to acquire handwriting skills?". Thus, we looked into three different research domains: education; pedagogic sciences; and social human-robot interaction (HRI). On the one hand, we investigated the types of errors commonly present in children’s handwriting. On the other hand, we needed to explore different educational theories to find alternate learning methods that would maximise children’s learning gains. In addition, we investigated various possibilities for integrating a social robot into educational scenarios and measured its impact on the children’s learning process. This thesis presents five studies that follow an incremental evolution of the research. The first three studies were based on the interaction between two children, in the presence of either a human or a robotic facilitator, with the children performing a collaborative writing activity. We explored the ‘peer-learning’ and ‘peer-tutoring’ methods and compared the differences in terms of interaction behaviour and learning gains. Further, we analysed children’s interpersonal distance and feelings of responsibility towards their peers, both in the presence of a robot and human facilitator. Finally, we examined children’s assessment behaviour towards their peers and self-disclosure towards the robotic facilitator. The outcomes of these first studies provided enough data to generate a taxonomy of children handwriting errors to be used in the subsequent studies. They hinted that the ‘peer-tutoring’ method had a positive influence on children’s learning and corrective feedback to their peers in the presence of the social robot, providing positive support to the applicability of social robots as ‘facilitators’ in handwriting learning. The next step was to study the effects of the robot in the peer-tutoring scenario when assuming the role of a peer, directly interacting and performing a collaborative writing activity with the children. Thus, we developed an educational system with an autonomous social behaviour of a robot exhibiting various levels of learning competencies. The tutor-child provides corrective feedback on the writing of the learner-robot and in turn, enhances his writing skills. The robot’s writing was generated by an algorithm incorporating human-inspired movements and could reproduce a set of writing errors from the error taxonomy. We tested the system by conducting two longitudinal studies, each of them composed of four sessions. In the fourth study, we assigned the robot two contrasting competencies: ‘learning’ and ‘non-learning’. We measured the differences in children’s learning gains and changes in their perceptions towards the learner-robot. The fifth study followed a similar interaction scenario and research questions, but this time the robot performed three learning competencies: ‘continuous-learning’;’non-learning’ and ‘personalised-learning’. The findings of these studies show that the children improved more with the robot exhibiting learning, continuous-learning and personalised-learning competencies compared to the robot showing non-learning competency. The way children’s learning and perception of the robot evolved over time confirmed the need for multiple sessions for the last two studies. This thesis presents some evidence that social robots can be successfully used as tools for enhancing children’s handwriting skills in peer-learning scenarios.

EPFL2019

Introducing Productive Engagement for Social Robots Supporting Learning

Jauwairia Nasir

We have all been one such student or seen such students who can maintain the 'good student' image while playing a video game under the table or those loyal backbenchers, seemingly always distracted, who then ace their exams. These intricacies of human behaviors are just a few examples of what makes it non-trivial and challenging even for expert teachers to know how students' visible behaviors relate with learning. As research investigates ways in which robots and AI can support teachers and students, it is faced with the same challenge of inferring students' engagement; thus, making the investigation of this topic increasingly popular in educational HRI. The state of the art usually explores the relationship between the robot behaviors and the engagement state of the learner while assuming a linear relationship between engagement and learning. However, is it correct to assume that to maximize learning, one needs to maximize engagement? Furthermore, conventional supervised engagement models require human annotators to get labels. This not only is laborious but can also introduce subjectivity. Can we have machine-learning engagement models where annotations do not rely on human annotators? Additionally, with the increase in open-ended learning activities which by design employ the 'learning by failing' paradigm, in-task performance can not be the best measure for learning. Can we instead rely on multi-modal behaviors? In an effort to cater for these challenges, this thesis dives deep to identify and quantify the relationship between learning and engagement, which we term as Productive Engagement (PE). In order to develop, design, and evaluate our PE framework, (1) we first designed and developed an open-ended collaborative learning activity that served as a platform for evaluating different robot variants over time. With 98 children interacting with the baseline version from 2 international Swiss schools, we showed that in-task performance and learning are indeed not correlated. Thus, this showed the importance of not being limited to robot interventions that affect only superficial measures of students' learning. (2) Then, with learner's multi-modal behaviors, we showed that indeed there is a hidden link between learner's behaviors and learning that can be quantified, i.e., validating the proposed concept of Productive Engagement. (3) This quantifiable link surfaced three collaborative multi-modal learner profiles, by using a forward and backward clustering and classification technique, two of which are linked to higher learning. This technique gave a possibility to surface data driven labels for engagement; thus, evading the process of human annotations. We then identified similarities and differences between these learner profiles both at an aggregate and at the temporal level. (4) Based on (3), we constructed a PE score that can either be directly used as an assessment metric by a social robot in real-time or as data driven labels for building more sophisticated regression models. (5) With the learner profiles and the PE score, we designed and evaluated more advanced robot variants for the final studies with ~160 students from 7 international Swiss schools. With the design of different robot variants that employ knowledge about the learner's skills conducive to learning, rather than domain knowledge, in order to provide interventions; we provided a complementary perspective on the role of social robots in educational settings.

EPFL2022

Continuous extraction of task constraints in a robot programming by demonstration framework

Sylvain Calinon

Robot Programming by Demonstration (RbD), also referred to as Learning by Imitation, explores user-friendly means of teaching a robot new skills. Recent advances in RbD have identified a number of key-issues for ensuring a generic approach to the transfer of skills across various agents and contexts. This thesis focuses on the two generic questions of what-to-imitate and how-to-imitate, which are respectively concerned with the problem of extracting the essential features of a task and determining a way to reproduce these essential features in different situations. The perspective adopted in this work is that a skill can be described efficiently at a trajectory level and that the robot may infer what are the important characteristics of the skill by observing multiple demonstrations of it, assuming that the important characteristics are invariant across the demonstrations. The proposed approach is based on the use of well-established statistical methods in a RbD application, by using Hidden Markov Model (HMM) as a first approach and then moving on to the joint use of Gaussian Mixture Model (GMM) and Gaussian Mixture Regression (GMR). Even if these methods were applied extensively in various fields of research including human motion analysis and robotics, their use essentially focused on gesture recognition rather than on gesture reproduction. Moreover, the models were usually trained offline using large datasets. Thus, the use and combination of these machine learning techniques in a Learning by Imitation framework is challenging and has not been extensively studied yet. In this thesis, we show that these methods are well suited for incremental and active teaching scenarios where the user only provides a small number of demonstrations, either by wearing a set of motions sensors attached to his/her body, or by helping the robot refine its skill by kinesthetic teaching, that is, by embodying the robot and putting it through the motion. These techniques are then applied for enabling a Fujitsu HOAP-2 and HOAP-3 humanoid robot to extract automatically different constraints by observing several manipulation skills and to reproduce these skills in various situations through Gaussian Mixture Regression (GMR). The contributions of this thesis are threefold: (1) it contributes to RbD by proposing a generic probabilistic framework to deal with recognition, generalization, reproduction and evaluation issues, and more specifically to deal with the automatic extraction of task constraints and with the determination of a controller satisfying several constraints simultaneously to reproduce the skill in a different context; (2) it contributes to Human-Robot Interaction (HRI) by proposing active teaching methods that puts the human teacher "in the loop" of the robot's learning by using an incremental scaffolding process and by using different modalities to produce the demonstrations; (3) it finally contributes to robotics and HRI through various real-world applications of the framework showing learning and reproduction of communicative gestures and manipulation skills. The generality of the proposed framework is also demonstrated through its joint use with other learning approaches in collaborative experiments.

EPFL2007