Juan Carlos Farah
The presence of conversational agents (or chatbots) in educational contexts has been steadily increasing over the past few years. Recent surveys have shown widespread interest in the use of chatbots in education, both for research and practice. Although these surveys highlight tangible benefits and promising future applications of educational chatbots, several challenges limit our ability to integrate these chatbots into educational contexts, including technological, pedagogical, interaction, and design challenges. In this thesis, we motivate our approach to these challenges by formulating one overarching research question: How can we guide the integration of chatbots into domain-specific learning contexts? To address this question and, in turn, tackle some of the challenges identified in the literature, we propose a conceptual framework spanning four different dimensions. Following the design-based research methodology, we address each dimension in a corresponding phase of our design process, undertaking multiple iterations of the design cycle within each phase.Our investigation starts by addressing the technological foundations of our framework through an application development architecture and a learning analytics pipeline aimed at supporting the creation of interactive applications for digital education platforms and providing access to the data generated when learners interact with these applications. Using our architecture, we then zoom in on one domain (software engineering education) and develop the applications needed to scaffold pedagogical scenarios in which these chatbots could interact with learners. Specifically, we propose the code review notebook, a template for building technopedagogical scenarios to support teaching programming best practices. Code review notebooks resemble the interactions developers have on social coding platforms and, given the popularity of chatbots on these platforms, are especially suitable for educational chatbots.In a series of online, observational, and field studies, we then explore different interaction strategies that could be harnessed by educational chatbots in their conversations with learners. In particular, we conducted three field studies to assess the effects that educational chatbots following (i) Wizard of Oz, (ii) rule-based, and (iii) large language model-based conversational strategies could have on different aspects of the learning experience. Findings from these studies are relevant to instructors looking to integrate educational chatbots into their practice and served to inform two final contributions proposed in this thesis. These contributions focus on design processes and comprise a model to guide the participatory design of educational chatbots, as well as a technical blueprint to define how these chatbots could be integrated into digital learning applications. Zooming out from the software engineering education use case, the two final contributions aim to generalize our findings to other educational domains.As application programming interfaces to powerful generative language models become widely accessible, we can only expect that increasingly complex educational chatbots will become ubiquitous in the years to come. Understanding how learners interact with these chatbots and providing the support necessary to guide their development is therefore of paramount importance. Our framework aligns itself closely with this line of research.
EPFL2023
