Automated human-level diagnosis of dysgraphia using a consumer tablet

The academic and behavioral progress of children is associated with the timely development of reading and writing skills. Dysgraphia, characterized as a handwriting learning disability, is usually associated with dyslexia, developmental coordination disorder (dyspraxia), or attention deficit disorder, which are all neuro-developmental disorders. Dysgraphia can seriously impair children in their everyday life and require therapeutic care. Early detection of handwriting difficulties is, therefore, of great importance in pediatrics. Since the beginning of the 20th century, numerous handwriting scales have been developed to assess the quality of handwriting. However, these tests usually involve an expert investigating visually sentences written by a subject on paper, and, therefore, they are subjective, expensive, and scale poorly. Moreover, they ignore potentially important characteristics of motor control such as writing dynamics, pen pressure, or pen tilt. However, with the increasing availability of digital tablets, features to measure these ignored characteristics are now potentially available at scale and very low cost. In this work, we developed a diagnostic tool requiring only a commodity tablet. To this end, we modeled data of 298 children, including 56 with dysgraphia. Children performed the BHK test on a digital tablet covered with a sheet of paper. We extracted 53 handwriting features describing various aspects of handwriting, and used the Random Forest classifier to diagnose dysgraphia. Our method achieved 96.6% sensibility and 99.2% specificity. Given the intra-rater and inter-rater levels of agreement in the BHK test, our technique has comparable accuracy for experts and can be deployed directly as a diagnostics tool.

The transferability of handwriting skills: from the Cyrillic to the Latin alphabet

Thibault Lucien Christian Asselborn, Wafa Monia Benkaouar Johal, Pierre Dillenbourg

Do handwriting skills transfer when a child writes in two different scripts, such as the Latin and Cyrillic alphabets? Are our measures of handwriting skills intrinsically bound to one alphabet or will a child who faces handwriting difficulties in one script experience similar difficulties in the other script? To answer these questions, 190 children from grades 1–4 were asked to copy a short text using both the Cyrillic and Latin alphabets on a digital tablet. A recent change of policy in Kazakhstan gave us an opportunity to measure transfer, as the Latin-based Kazakh alphabet has not yet been introduced. Therefore, pupils in grade 1 had a 6-months experience in Cyrillic, and pupils in grades 2, 3, and 4 had 1.5, 2.5, and 3.5 years of experience in Cyrillic, respectively. This unique situation created a quasi-experimental situation that allowed us to measure the influence of the number of years spent practicing Cyrillic on the quality of handwriting in the Latin alphabet. The results showed that some of the differences between the two scripts were constant across all grades. These differences thus reflect the intrinsic differences in the handwriting dynamics between the two alphabets. For instance, several features related to the pen pressure on the tablet are quite different. Other features, however, revealed decreasing differences between the two scripts across grades. While we found that the quality of Cyrillic writing increased from grades 1–4, due to increased practice, we also found that the quality of the Latin writing increased as well, despite the fact that all of the pupils had the same absence of experience in writing in Latin. We can therefore interpret this improvement in Latin script as an indicator of the transfer of fine motor control skills from Cyrillic to Latin. This result is especially surprising given that one could instead hypothesize a negative transfer, i.e., that the finger controls automated for one alphabet would interfere with those required by the other alphabet. One interesting side-effect of these findings is that the algorithms that we developed for the diagnosis of handwriting difficulties among French-speaking children could be relevant for other alphabets, paving the way for the creation of a cross-lingual model for the detection of handwriting difficulties.

2021

The Dynamics of Handwriting Improves the Automated Diagnosis of Dysgraphia

Thibault Lucien Christian Asselborn, Wafa Monia Benkaouar Johal, Pierre Dillenbourg

Handwriting disorder (termed dysgraphia) is a far from a singular problem as nearly 8.6% of the population in France is considered dysgraphic. Moreover, research highlights the fundamental importance to detect and remediate these handwriting difficulties as soon as possible as they may affect a child's entire life, undermining performance and self-confidence in a wide variety of school activities. At the moment, the detection of handwriting difficulties is performed through a standard test called BHK. This detection, performed by therapists, is laborious because of its high cost and subjectivity. We present a digital approach to identify and characterize handwriting difficulties via a Recurrent Neural Network model (RNN). The child under investigation is asked to write on a graphics tablet all the letters of the alphabet as well as the ten digits. Once complete, the RNN delivers a diagnosis in a few milliseconds and demonstrates remarkable efficiency as it correctly identifies more than 90% of children diagnosed as dysgraphic using the BHK test. The main advantage of our tablet-based system is that it captures the dynamic features of writing -- something a human expert, such as a teacher, is unable to do. We show that incorporating the dynamic information available by the use of tablet is highly beneficial to our digital test to discriminate between typically-developing and dysgraphic children.

2019

Learning (Good Handwriting In Greek) By Teaching (A Humanoid Robot)

Thibault Lucien Christian Asselborn, Wafa Monia Benkaouar Johal, Thanasis Hadzilacos

We report on a follow-up study of the Co-Writer project at EPFL [1]; we confirm their findings, extend the applicability to another language with a different alphabet (Greek) and go into an in-depth qualitative study of the child-robot relationship. The core idea of "learning by teaching" is that the student learns by undertaking to teach what they are supposed to learn which motivates them into ownership of the process. Although a highly effective method [2], there are serious practical obstacles in scaling learning by teaching to larger numbers or extending it to younger learners and to certain skills such as handwriting. This is where we enlist the assistance of a humanoid robot to help a young learner with her handwriting by assuming the role of the cacographer pupil who asks help from its human friend. The robot pretends writing mistakes similar to the child's own; the child recognizes the errors as such even when she would refuse to acknowledge them in her own writing, and makes an extra effort for better writing as the robot's instructor; the robot improves its writing; the child goes through another round of amelioration; and, hopefully, gets in the habit of better writing. In order to extend the original study for the Greek language and alphabet we first needed to collect data from beginner writers, children of 6-9 years. The nine-month study for collecting and evaluating handwriting samples from primary school students in Cyprus in order to examine and describe cacography for the Greek alphabet used a questionnaire, an evaluation sheet for the letter formation and copies of students' notebooks. It involved four primary schools in Limassol, Cyprus and a total of 68 students and 11 teachers. We were thus able to obtain a consensus on what constitutes poor letter writing in Greek. All writing samples were evaluated by teachers and were classified based on Chandra et al. [3] taxonomy. The qualitative research involved three case studies of children who held two sessions each with the humanoid programmable robot NAO. We video recorded the child-robot interactions and assessed their progress in handwriting. In all three cases, albeit not in the same degree, we saw that (a) the child-robot interaction quickly improved; the increasing number of correct remarks from all children in only two meetings shows the comfort that they gained with lab environment and the robot itself and (b) the correctness of children remarks and their comments about letters shows their ability to distinguish the correct from the wrong letters and to justify their answers. Both conclusions confirm the findings of the Co-Writer project [4].

IATED-INT ASSOC TECHNOLOGY EDUCATION & DEVELOPMENT2020

The transferability of handwriting skills: from the Cyrillic to the Latin alphabet

Thibault Lucien Christian Asselborn, Wafa Monia Benkaouar Johal, Pierre Dillenbourg

2021

Learning (Good Handwriting In Greek) By Teaching (A Humanoid Robot)

Thibault Lucien Christian Asselborn, Wafa Monia Benkaouar Johal, Thanasis Hadzilacos

IATED-INT ASSOC TECHNOLOGY EDUCATION & DEVELOPMENT2020