Analysis and Remediation of Handwriting difficulties

Even with correct training, up to 25% of children never master handwriting like their peers. While research shows a correlation between handwriting difficulties and school failure, these difficulties can also impact children in their self esteem and behavioral development. Since the mastery of handwriting requires a lot of different skills, it is never easy to understand where a given child is facing difficulties, nor how exactly to help him/her overcome them. For this reason, it is of prime importance to detect and understand childrenâs handwriting difficulties the earliest possible in order to propose the most effective remediation possible. In this thesis, we first introduce a modernize version of the currently adopted handwriting tests, that show clear limitations in the era of digitalization. Indeed, the nature itself of these tests, conducted on paper, restricts them to the analysis of the final static aspect of handwriting. Its dynamics, found to be very important, is therefore hidden and cannot be taken into consideration. For this reason, we designed in collaboration with therapists several features that describe different aspects of handwriting, which are not limited to static but also capture kinematic, pressure and tilt. The designed features have the main advantage to describe very low level aspects of handwriting, which makes them quite independent of the writing content. We verified this hypothesis by giving the proof of concept that our model for automatic detection of handwriting difficulties can be translated from the latin to the the cyrillic alphabet. In the same way, we demonstrated that our model can also, given retraining, be used on paper or directly on digital tablets, like iPads. Finally, we introduced our iPad-based test allowing to extract the multidimensional handwriting profile of the child. This test aims to answer the first of the afore-discussed problems, by allowing to extract the specific strengths and weaknesses of a child, in less than a minute, on different aspects and at different granularities.

The second part of this thesis tackles the problem of designing remediation activities for handwriting difficulties. We designed activities specifically targeting the handwriting aspects identified by the model and obtained a preliminary proof of concept that serious games targeting specific skills of handwriting (e.g. pressure, kinematic, tilt, ...) can have a positive impact on the overall quality of handwriting. Finally the two last chapters of this thesis tackle two corollary, but still crucial questions related to handwriting remediation. After the integration of these remediation activities in a Child-Robot Interaction scenario, in which the child is the teacher of the robot, we gave the proof of concept of the importance of the design of robot behaviors towards social acceptance with children, something especially important knowing the importance of the childâs perception of the robot and the interaction with it in such a scenario. Finally, in the last Chapter, we investigated whether it is possible to "remediate some handwriting difficulties by preventing them", i.e., by supporting pre-school children in the acquisition of the fundamental visuo-motor coordination skills required by handwriting.

When Children Teach a Robot to Write: An Autonomous Teachable Humanoid Which Uses Simulated Handwriting

Pierre Dillenbourg, Deanna Hood, Séverin Lemaignan

This article presents a novel robotic partner which children can teach handwriting. The system relies on the learning by teaching paradigm to build an interaction, so as to stimulate meta-cognition, empathy and increased self-esteem in the child user. We hypothesise that use of a humanoid robot in such a system could not just engage an unmotivated student, but could also present the opportunity for children to experience physically-induced benefits encountered during human-led handwriting interventions, such as motor mimicry. By leveraging simulated handwriting on a synchronised tablet display, a nao humanoid robot with limited fine motor capabilities has been configured as a suitably embodied handwriting partner. Statistical shape models derived from principal component analysis of a dataset of adult-written letter trajectories allow the robot to draw purposefully deformed letters. By incorporating feedback from user demonstrations, the system is then able to learn the optimal parameters for the appropriate shape models. Preliminary in situ studies have been conducted with primary school classes to obtain insight into children’s use of the novel system. Children aged 6-8 successfully engaged with the robot and improved its writing to a level which they were satisfied with. The validation of the interaction represents a significant step towards an innovative use for robotics which addresses a widespread and socially meaningful challenge in education.

2015

Control of Bio-Inspired Sprawling Posture Quadruped Robots with an Actuated Spine

Tomislav Horvat

Sprawling posture robots are characterized by upper limb segments protruding horizontally from the body, resulting in lower body height and wider support on the ground. Combined with an actuated segmented spine and tail, such morphology resembles that of salamanders or crocodiles. Although bio-inspired salamander-like robots with simple rotational limbs have been created, not much research has been done on kinematically redundant bio-mimetic robots that can closely replicate kinematics of sprawling animal gaits. Being bio-mimetic could allow a robot to have some of the locomotion skills observed in those animals, expanding its potential applications in challenging scenarios. At the same time, the robot could be used to answer questions about the animal's locomotion. This thesis is focused on developing locomotion controllers for such robots. Due to their high number of degrees of freedom (DoF), the control is based on solving the limb and spine inverse kinematics to properly coordinate different body parts. It is demonstrated how active use of a spine improves the robot's walking and turning performance. Further performance improvement across a variety of gaits is achieved by using model predictive control (MPC) methods to dictate the motion of the robot's center of mass (CoM). The locomotion controller is reused on an another robot (OroBOT) with similar morphology, designed to mimic the kinematics of a fossil belonging to Orobates, an extinct early tetrapod. Being capable of generating different gaits and quantitatively measuring their characteristics, OroBOT was used to find the most probable way the animal moved. This is useful because understanding locomotion of extinct vertebrates helps to conceptualize major transitions in their evolution. To tackle field applications, e.g. in disaster response missions, a new generation of field-oriented sprawling posture robots was built. The robustness of their initial crocodile-inspired design was tested in the animal's natural habitat (Uganda, Africa) and subsequently enhanced with additional sensors, cameras and computer. The improvements to the software framework involved a smartphone user interface visualizing the robot's state and camera feed to improve the ease of use for the operator. Using force sensors, the locomotion controller is expanded with a set of reflex control modules. It is demonstrated how these modules improve the robot's performance on rough and unstructured terrain. The robot's design and its low profile allow it to traverse low passages. To also tackle narrow passages like pipes, an unconventional crawling gait is explored. While using it, the robot lies on the ground and pushes against the pipe walls to move the body. To achieve such a task, several new control and estimation modules were developed. By exploring these problems, this thesis illustrates fruitful interactions that can take place between robotics, biology and paleontology.

EPFL2018

Learning How to Write with a Social Robot

Shruti Chandra

Learning handwriting is one of the main occupations for children in primary education. It is a complex skill that has been associated with children’s learning and development. Poor handwriting skills affect children negatively concerning their academic performance and motivation. In schools, children with handwriting problems need extra support and one-to-one interventions from teachers. Personalised one-to-one support requires additional resources that most schools generally cannot afford. Therefore, there is a need to explore different methods of learning and contemporary technologies to develop new educational tools to foster children’s learning process of handwriting. The research presented in this thesis addresses the question of - “How to create a social robot that can help children to acquire handwriting skills?". Thus, we looked into three different research domains: education; pedagogic sciences; and social human-robot interaction (HRI). On the one hand, we investigated the types of errors commonly present in children’s handwriting. On the other hand, we needed to explore different educational theories to find alternate learning methods that would maximise children’s learning gains. In addition, we investigated various possibilities for integrating a social robot into educational scenarios and measured its impact on the children’s learning process. This thesis presents five studies that follow an incremental evolution of the research. The first three studies were based on the interaction between two children, in the presence of either a human or a robotic facilitator, with the children performing a collaborative writing activity. We explored the ‘peer-learning’ and ‘peer-tutoring’ methods and compared the differences in terms of interaction behaviour and learning gains. Further, we analysed children’s interpersonal distance and feelings of responsibility towards their peers, both in the presence of a robot and human facilitator. Finally, we examined children’s assessment behaviour towards their peers and self-disclosure towards the robotic facilitator. The outcomes of these first studies provided enough data to generate a taxonomy of children handwriting errors to be used in the subsequent studies. They hinted that the ‘peer-tutoring’ method had a positive influence on children’s learning and corrective feedback to their peers in the presence of the social robot, providing positive support to the applicability of social robots as ‘facilitators’ in handwriting learning. The next step was to study the effects of the robot in the peer-tutoring scenario when assuming the role of a peer, directly interacting and performing a collaborative writing activity with the children. Thus, we developed an educational system with an autonomous social behaviour of a robot exhibiting various levels of learning competencies. The tutor-child provides corrective feedback on the writing of the learner-robot and in turn, enhances his writing skills. The robot’s writing was generated by an algorithm incorporating human-inspired movements and could reproduce a set of writing errors from the error taxonomy. We tested the system by conducting two longitudinal studies, each of them composed of four sessions. In the fourth study, we assigned the robot two contrasting competencies: ‘learning’ and ‘non-learning’. We measured the differences in children’s learning gains and changes in their perceptions towards the learner-robot. The fifth study followed a similar interaction scenario and research questions, but this time the robot performed three learning competencies: ‘continuous-learning’;’non-learning’ and ‘personalised-learning’. The findings of these studies show that the children improved more with the robot exhibiting learning, continuous-learning and personalised-learning competencies compared to the robot showing non-learning competency. The way children’s learning and perception of the robot evolved over time confirmed the need for multiple sessions for the last two studies. This thesis presents some evidence that social robots can be successfully used as tools for enhancing children’s handwriting skills in peer-learning scenarios.