Gaze Analysis methods for Learning Analytics

Eye-tracking had been shown to be predictive of expertise, task-based success, task-difficulty, and the strategies involved in problem solving, both in the individual and collaborative settings. In learning analytics, eye-tracking could be used as a powerful tool, not only to differentiate between the levels of expertise and task-outcome, but also to give constructive feedback to the users. In this dissertation, we show how eye-tracking could prove to be useful to understand the cognitive processes underlying dyadic interaction; in two contexts: pair program comprehension and learning with a Massive Open Online Course (MOOC). The first context is a typical collaborative work scenario, while the second is a special case of dyadic interaction namely the teacher-student pair. We also demonstrate, using one example experiment, how the findings about the relation between the learning outcome in MOOCs and the students' gaze patterns can be leveraged to design a feedback tool to improve the students' learning outcome and their attention levels while learning through a MOOC video. We also show that the gaze can also be used as a cue to resolve the teachers' verbal references in a MOOC video; and this way we can improve the learning experiences of the MOOC students. This thesis is comprised of five studies. The first study, contextualised within a collaborative setting, where the collaborating partners tried to understand the given program. In this study, we examine the relationship among the gaze patterns of the partners, their dialogues and the levels of understanding that the pair attained at the end of the task. The next four studies are contextualised within the MOOC environment. The first MOOC study explores the relationship between the students' performance and their attention level. The second MOOC study, which is a dual eye-tracking study, examines the relation between the individual and collaborative gaze patterns and their relation with the learning outcome. This study also explores the idea of activating students' knowledge, prior to receiving any learning material, and the effect of different ways to activate the students' knowledge on their gaze patterns and their learning outcomes. The third MOOC study, during which we designed a feedback tool based on the results of the first two MOOC studies, demonstrates that the variables we proposed to measure the students' attention, could be leveraged upon to provide feedback about their gaze patterns. We also show that using this feedback tool improves the students' learning outcome and their attention levels. The fourth and final MOOC study shows that augmenting a MOOC video with the teacher's gaze information helps improving the learning experiences of the students. When the teacher's gaze is displayed the perceived difficulty of the content decreases significantly as compared to the moments when there is no gaze augmentation. In a nutshell, through this dissertation, we show that the gaze can be used to understand, support and improve the dyadic interaction, in order to increase the chances of achieving a higher level of task-based success.

Eye-tracking and artificial intelligence to enhance motivation and learning

Pierre Dillenbourg, Kshitij Sharma

The interaction with the various learners in a Massive Open Online Course (MOOC) is often complex. Contemporary MOOC learning analytics relate with click-streams, keystrokes and other user-input variables. Such variables however, do not always capture users' learning and behavior (e.g., passive video watching). In this paper, we present a study with 40 students who watched a MOOC lecture while their eye-movements were being recorded. We then proposed a method to define stimuli-based gaze variables that can be used for any kind of stimulus. The proposed stimuli-based gaze variables indicate students' content-coverage (in space and time) and reading processes (area of interest based variables) and attention (i.e., with-me-ness), at the perceptual (following teacher's deictic acts) and conceptual levels (following teacher discourse). In our experiment, we identified a significant mediation effect of the content coverage, reading patterns and the two levels of with-me-ness on the relation between students' motivation and their learning performance. Such variables enable common measurements for the different kind of stimuli present in distinct MOOCs. Our long-term goal is to create student profiles based on their performance and learning strategy using stimuli-based gaze variables and to provide students gaze-aware feedback to improve overall learning process. One key ingredient in the process of achieving a high level of adaptation in providing gaze-aware feedback to the students is to use Artificial Intelligence (AI) algorithms for prediction of student performance from their behaviour. In this contribution, we also present a method combining state-of-the-art AI technique with the eye-tracking data to predict student performance. The results show that the student performance can be predicted with an error of less than 5%.

SPRINGER HEIDELBERG2020

Expanding Eye-Tracking Methods to Explain the Socio-Cognitive Effects of Shared Annotations

Andrea Mazzei

Social technologies are leading to transformations in education, by empowering the way learners connect to each other, by introducing new means for teaching and learning and by reshaping the way knowledge is delivered. Annotating texts is a learning strategy that is spontaneously adopted by learners during lectures or while reading text books. Students still individually take annotations on paper, that tend to be thrown away when the learning goal is achieved. Students also engage occasionally in the spontaneous practice of note sharing. Our work explores experimentally the benefits of note-taking and note-sharing behaviour. First, we study how sharing student-annotated instructional texts can improve learning, enriching an experimental approach with a new eye tracking method. Second, we conduct experiments on computer-mediated note sharing in the classroom. Our results demonstrate the virtuous circle of note-taking: both annotating while reading and reading annotated documents lead to a better learning achievement. In the first experimental study, we measure if the presence of annotations, on an instruc- tional text, can influence the reading pattern, and how visual features of annotations can elicit the reader’s attention. To complement the results concerning learning and reading comprehension, we look into the readers’ gaze patterns, to explain how the students’ learning outcome relates to reading annotated texts. For this purpose, we design a novel eye tracker, that can be used to study reading and note taking in unconstrained experimental settings. This eye tracking system is based on a systematic error correction procedure that exploits the appearance similarity between the annotated texts and the spatial distribution of the fixation points. We show that this method can extract accurate gaze measures without introducing experimental constraints, that could disturb the note taking process and affect the readers’ comprehension. In the second study, we move from a controlled experimental setting to the classroom. We discuss how the use of technology can facilitate a spontaneous transition from personal to shared annotations and support students in the learning process. We complement the analysis by reporting a friendship bias in browsing the shared annotated material. We further speculate on the potential of shared annotations in triggering adaptations of the instructional material and teaching workflow. These two studies provided an insightful understanding of the effects of student-generated annotations on reading comprehension, and the underlying impact on a community of learners. The obtained findings should inspire further experimentation on social learning environments, meant to facilitate knowledge sharing through shared annotations and their diffusion within educational institutions.

EPFL2013

Learning Analytics for Adaptive and Self-Improving Learning Environments for Inductive Teaching

Louis Pierre Faucon

The emergence of digital technology is changing education in many ways. A particularly interesting aspect of this transformation is the development of learning environments that can automatically adapt to individual students and can collect data in order to automatically improve themselves. How a learning environment should adapt to students is dependent on the pedagogical approach. In this thesis, we contribute to adaptive and self-improving learning environments that support students during inductive reasoning activities. Inductive teaching is a pedagogical approach that yields very beneficial outcomes for students, but has not received sufficient technological support. Using inductive reasoning, students infer general rules or concepts from observations. This pedagogical approach is motivated by the idea that learning outcomes will increase when students construct the knowledge by themselves.First, we contribute to student modeling and self-improvement for learning environments. We share the results of a study that we conducted using data collected in several large classrooms. We develop a mechanism that collects data, estimates the progress of students and predicts in real-time their future progress. The goal of our approach is to aggregate the predictions to support adaptive decisions by teachers in classrooms. Moreover, we share the results of a second study with MOOC students. We use a generative model, based on a Semi-Markov Chain, to model and simulate sequences of actions taken by students on the MOOC platform. Additionally, based on observations of students from arbitrary Bayesian models, we propose and analyse a self-improving algorithm relying on Thompson Sampling optimisation that maximizes students learning outcomes over time.Second, we contribute to the analysis of inductive teaching and individual differences in inductive reasoning. We report results on an experiment with 222 students solving tasks of categorisation of images. Our study revealed that students individually differ in how they choose to classify examples based on feature differences between the examples. We define the individual differences of students as their inductive bias. This result motivates the importance of adaptivity in the selection of examples during inductive learning activities. Additionally, we analyse how much students change their inductive bias when confronted to negative feedback. We find that students are influenced by the feedback, but that this influence decays after a short period of time.Third, we contribute multiple algorithms that constitute together the necessary components of an adaptive and self-improving learning environment for inductive teaching. Notably, we designed algorithms for a learning environment to extract representations of students' biases from data, estimate and trace individual students' biases, and to optimally select personalised examples for an inductive learning activity.Finally, we conclude by describing the concept of probabilistic testing as a promising assessment mechanism for inductive learning environments. We provide preliminary observations and theoretical results for the use of probabilistic testing in the context of adaptive inductive teaching. In particular, by using probabilistic tests, a learning environment can estimate more quickly students' inductive biases.