Discovering Interaction Patterns in Online Learning Environments

The increasing amount of data collected in online learning environments provides unique opportunities to better understand the learning processes in different educational settings. Learning analytics research aims at understanding and optimizing learning and the environments in which it occurs. A crucial step towards this goal is to adapt and develop adequate computational methods to process and analyze the learning-related data and to present information in intelligible ways to educational stakeholders.

In this thesis we investigate interaction patterns of learners in two different online learning environments: Massive Open Online Courses (MOOCs) and Realto, an online platform for integrated Vocational Education and Training (VET). We analyze interaction patterns across three principal dimensions: time, activity, and social. To obtain a better understanding of the complex learning behaviours, it is essential to consider these different aspects of the educational data. We develop novel methods and use existing techniques from sequential pattern mining, content analysis, and social network analysis to model and track interaction patterns of learners across the three mentioned dimensions.

As regards the time dimension, we present methods to model temporal patterns of learners' participation. We introduce novel techniques to discover and quantify online regularity in terms of following a certain daily or weekly time schedule. We investigate the relation between students' regularity level and their performance in a MOOC course.

Concerning the activity dimension, we analyze learners' activity sequences in order to identify and track the evolution of their study approaches over time. By clustering study pattern sequences in a MOOC course, we extract different engagement profiles among learners and describe their properties. Furthermore, we propose a complete processing pipeline for the unsupervised discovery of study patterns from sequential interaction logs. This pipeline is applicable at different levels of actions granularity and time resolution and enables to perform temporal analysis of learners' interaction patterns throughout the course duration.

For the social dimension, we explore the attributes of social interactions among learners. In the MOOC context, we combine content and social network analyses to study dynamics of forum discussions and the evolution of students' roles over time. In the context of Realto, we employ social network analysis to model the social interactions among learners and to study the structure of Realto-mediated communication among different stakeholders in the VET system.

Using the presented analytic methods, we provide novel insights into the interaction patterns of learners in MOOCs and Realto. Moreover, we present an implementation of these methods into an analytics dashboard for Realto researchers.

Social Network Analysis for Automatic Role Recognition

Sarah Favre

The computing community has shown a significant interest for the analysis of social interactions in the last decade. Different aspects of social interactions have been studied such as dominance, emotions, conflicts, etc. However, the recognition of roles has been neglected whereas these are a key aspect of social interactions. In fact, sociologists have shown not only that people play roles each time they interact, but also that roles shape behavior and expectations of interacting participants. The aim of this thesis is to fill this gap by investigating the problem of automatic role recognition in a wide range of interaction settings, including production environments, e.g. news and talk-shows, and spontaneous exchanges, e.g. meetings. The proposed role recognition approach includes two main steps. The first step aims at representing the individuals involved in an interaction with feature vectors accounting for their relationships with others. This step includes three main stages, namely segmentation of audio into turns (i.e. time intervals during which only one person talks), conversion of the sequence of turns into a social network, and use of the social network as a tool to extract features for each person. The second step uses machine learning methods to map the feature vectors into roles. The experiments have been carried out over roughly 90 hours of material. This is not only one of the largest databases ever used in literature on role recognition, but also the only one, to the best of our knowledge, including different interaction settings. In the experiments, the accuracy of the percentage of data correctly labeled in terms of roles is roughly 80% in production environments and 70% in spontaneous exchanges (lexical features have been added in the latter case). The importance of roles has been assessed in an application scenario as well. In particular, the thesis shows that roles help to segment talk-shows into stories, i.e. time intervals during which a single topic is discussed, with satisfactory performance. The main contributions of this thesis are as follows: To the best of our knowledge, this is the first work where social network analysis is applied to automatic analysis of conversation recordings. This thesis provides the first quantitative measure of how much roles constrain conversations, and a large corpus of recordings annotated in terms of roles. The results of this work have been published in one journal paper, and in five conference articles.

EPFL2011

Discovery and Temporal Analysis of Latent Study Patterns from MOOC Interaction Sequences

Pierre Dillenbourg, Mina Shirvani Boroujeni

Capturing students' behavioral patterns through analysis of sequential interaction logs, is an important task in educational data mining to enable more effective personalized support during the learning process. This study aims at discovery and temporal analysis of learners' study patterns in MOOC assessment periods. We address this problem using two different methods. First, following a pattern-driven approach, we identify learners' study patterns based on their interaction with video lectures and assignments. Through unsupervised clustering of study pattern sequences, we capture different longitudinal engagement profiles among learners in a MOOC course. Second, we propose temporal clustering framework for unsupervised discovery of latent patterns in learners' interaction data. We model and cluster activity sequences at each time step, and perform cluster matching to enable tracking learning behaviours over time. Our proposed pipeline is general and can be adopted for modeling and temporal analysis of interaction data at different levels of granularity, in various learning environments including MOOCs and Intelligent Tutoring Systems (ITS). We demonstrate the application of our proposed pipeline for detecting latents study patterns in a MOOC course.

2018

Discovery and Temporal Analysis of MOOC Study Patterns

Pierre Dillenbourg, Mina Shirvani Boroujeni

The large-scale and granular interaction data collected in online learning platforms such as massive open online courses (MOOCs) provide unique opportunities to better understand individuals' learning processes and could facilitate the design of personalized and more effective support mechanisms for learners. In this paper, we present two different methods of extracting study patterns from activity sequences. Unlike most of the previous works, with post hoc analysis of activity patterns, our proposed methods could be deployed during the course and enable the learners to receive real-time support and feedback. In the first method, following a hypothesis-driven approach, we extract predefined patterns from learners' interactions with the course materials. We then identify and analyze different longitudinal profiles among learners by clustering their study pattern sequences during the course. Our second method is a data-driven approach to discover latent study patterns and track them over time in a completely unsupervised manner. We propose a clustering pipeline to model and cluster activity sequences at each time step and then search for matching clusters in previous steps to enable tracking over time. The proposed pipeline is general and allows for analysis at different levels of action granularity and time resolution in various online learning environments. Experiments with synthetic data show that our proposed method can accurately detect latent study patterns and track changes in learning behaviours. We demonstrate the application of both methods on a MOOC dataset and study the temporal dynamics of learners' behaviour in this context.

2019

Social Network Analysis for Automatic Role Recognition

Sarah Favre

EPFL2011