A Speech-based Productive Engagement Metric for Real-time Human-Robot Interaction in Collaborative Educational Contexts

One of the major challenges in educational contexts is to correctly interpret the student's learning process in real-time, which is a necessary pre-requisite for timely and appropriate interventions. In educational HRI, multiple solutions have been proposed to endow robots with this key ability, typically relying on observable proxies such as in-task performance, affective behavior, engagement with the robot, etc. In this methodology paper, we propose and validate a metric for the real-time analysis of the behaviour of learners, allowing to assess whether they are engaged in meaningful learning behaviours. Specifically, building on the previously proposed concept of Productive Engagement, that inherently links learning with engagement, we hereby propose methods to quantify and compute it reliably in real-time. The training and testing of the methods is done using the open access PE-HRI-temporal dataset, that provides team level multi-modal temporal behavioral data, built from a study done with 68 students (34 teams) in a collaborative human-human educational activity mediated by a robot.