Bringing letters to life: handwriting with haptic-enabled tangible robots

In this paper, we present a robotic approach to improve the teaching of handwriting using the tangible, haptic-enabled and classroom-friendly Cellulo robots. Our efforts presented here are in line with the philosophy of the Cellulo platform: we aim to create a ready-to-use tool (i.e. a set of robot-assisted activities) to be used for teaching handwriting, one that is to coexist harmoniously with traditional tools and will contribute new added values to the learning process, complementing existing teaching practices. To maximize our potential contributions to this learning process, we focus on two promising aspects of handwriting: the visual perception and the visual-motor coordination. These two aspects enhance in particular two sides of the representation of letters in the mind of the learner: the shape of the letter (the grapheme) and the way it is drawn, namely the dynamics of the letter (the ductus). With these two aspects in mind, we do a detailed content analysis for the process of learning the representation of letters, which leads us to discriminate the specific skills involved in letter representation. We then compare our robotic method with traditional methods as well as with the combination of the two methods, in order to discover which of these skills can benefit from the use of Cellulo. As handwriting is taught from age 5, we conducted our experiments with 17 five-year-old children in a public school. Results show a clear potential of our robot-assisted learning activities, with a visible improvement in certain skills of handwriting, most notably in creating the ductus of the letters, discriminating a letter among others and in the average handwriting speed. Moreover, we show that the benefit of our learning activities to the handwriting process increases when it is used after traditional learning methods. These results lead to the initial insights into how such a tangible robotic learning technology may be used to create cost-effective collaborative scenarios for the learning of handwriting.

Cellulo: Tangible Haptic Swarm Robots for Learning

Ayberk Özgür

Robots are steadily becoming one of the significant 21st century learning technologies that aim to improve education within both formal and informal environments. Such robots, called Robots for Learning, have so far been utilized as constructionist tools or social agents that aided learning from distinct perspectives. This thesis presents a novel approach to Robots for Learning that aims to explore new added values by means of investigating uses for robots in educational scenarios beyond those that are commonly tackled: We develop a platform from scratch to be "as versatile as pen and paper", namely as composed of easy to use objects that feel like they belong in the learning ecosystem while being seamlessly usable across many activities that help teach a variety of subjects. Following this analogy, we design our platform as many low-cost, palm-sized tangible robots that operate on printed paper sheets, controlled by readily available mobile computers such as smartphones or tablets. From the learners' perspective, our robots are thus physical and manipulable points of hands-on interaction with learning activities where they play the role of both abstract and concrete objects that are otherwise not easily represented. We realize our novel platform in four incremental phases, each of which consists of a development stage and multiple subsequent validation stages. First, we develop accurately positioned tangibles, characterize their localization performance and test the learners' interaction with our tangibles in a playful activity. Second, we integrate mobility into our tangibles and make them full-blown robots, characterize their locomotion performance and test the emerging notion of moving vs. being moved in a learning activity. Third, we enable haptic feedback capability on our robots, measure their range of usability and test them within a complete lesson that highlights this newly developed affordance. Fourth, we develop the means of building swarms with our haptic-enabled tangible robots and test the final form of our platform in a lesson co-designed with a teacher. Our effort thus contains the participation of more than 370 child learners over the span of these phases, which leads to the initial insights into this novel Robots for Learning avenue. Besides its main contributions to education, this thesis further contributes to a range of research fields related to our technological developments, such as positioning systems, robotic mechanism design, haptic interfaces and swarm robotics.

EPFL2018

Learning to Control Planar Hitting Motions of a Robotic Arm in a Mini-Golf-like Task

Klas Jonas Alfred Kronander

In this thesis we tackle the problem of goal-oriented adaptation of a robot hitting motion. We propose the parameters that must be learned in order to use and adapt a basic hitting motion to play minigolf. Then, two different statistical methods are used to learn these parameters. The two methods are evaluated and compared. To validate the proposed approach, a minigolf control module is developed for a robotic arm. Using the different learning techniques, we show that a robot can learn the non-trivial task of deciding how the ball should be hit for a given position on a minigolf field. The result is a robust minigolf-playing system that outperforms most human players using only a small set of training examples.

2010

Imitation Learning of Motion Coordination in Robots

Elena Gribovskaya

The ease with which humans coordinate all their limbs is fascinating. Such a simplicity is the result of a complex process of motor coordination, i.e. the ability to resolve the biomechanical redundancy in an efficient and repeatable manner. Coordination enables a wide variety of everyday human activities from filling in a glass with water to pair figure skating. Therefore, it is highly desirable to endow robots with similar skills. Despite the apparent diversity of coordinated motions, all of them share a crucial similarity: these motions are dictated by underlying constraints. The constraints shape the formation of the coordination patterns between the different degrees of freedom. Coordination constraints may take a spatio-temporal form; for instance, during bimanual object reaching or while catching a ball on the fly. They also may relate to the dynamics of the task; for instance, when one applies a specific force profile to carry a load. In this thesis, we develop a framework for teaching coordination skills to robots. Coordination may take different forms, here, we focus on teaching a robot intra-limb and bimanual coordination, as well as coordination with a human during physical collaborative tasks. We use tools from well-established domains of Bayesian semiparametric learning (Gaussian Mixture Models and Regression, Hidden Markov Models), nonlinear dynamics, and adaptive control. We take a biologically inspired approach to robot control. Specifically, we adopt an imitation learning perspective to skill transfer, that offers a seamless and intuitive way of capturing the constraints contained in natural human movements. As the robot is taught from motion data provided by a human teacher, we exploit evidence from human motor control of the temporal evolution of human motions that may be described by dynamical systems. Throughout this thesis, we demonstrate that the dynamical system view on movement formation facilitates coordination control in robots. We explain how our framework for teaching coordination to a robot is built up, starting from intra-limb coordination and control, moving to bimanual coordination, and finally to physical interaction with a human. The dissertation opens with the discussion of learning discrete task-level coordination patterns, such as spatio-temporal constraints emerging between the two arms in bimanual manipulation tasks. The encoding of bimanual constraints occurs at the task level and proceeds through a discretization of the task as sequences of bimanual constraints. Once the constraints are learned, the robot utilizes them to couple the two dynamical systems that generate kinematic trajectories for the hands. Explicit coupling of the dynamical systems ensures accurate reproduction of the learned constraints, and proves to be crucial for successful accomplishment of the task. In the second part of this thesis, we consider learning one-arm control policies. We present an approach to extracting non-linear autonomous dynamical systems from kinematic data of arbitrary point-to-point motions. The proposed method aims to tackle the fundamental questions of learning robot coordination: (i) how to infer a motion representation that captures a multivariate coordination pattern between degrees of freedom and that generalizes this pattern to unseen contexts; (ii) whether the policy learned directly from demonstrations can provide robustness against spatial and temporal perturbations. Finally, we demonstrate that the developed dynamical system approach to coordination may go beyond kinematic motion learning. We consider physical interactions between a robot and a human in situations where they jointly perform manipulation tasks; in particular, the problem of collaborative carrying and positioning of a load. We extend the approach proposed in the second part of this thesis to incorporate haptic information into the learning process. As a result, the robot adapts its kinematic motion plan according to human intentions expressed through the haptic signals. Even after the robot has learned the task model, the human still remains a complex contact environment. To ensure robustness of the robot behavior in the face of the variability inherent to human movements, we wrap the learned task model in an adaptive impedance controller with automatic gain tuning. The techniques, developed in this thesis, have been applied to enable learning of unimanual and bimanual manipulation tasks on the robotics platforms HOAP-3, KATANA, and i-Cub, as well as to endow a pair of simulated robots with the ability to perform a manipulation task in the physical collaboration.