Using Educational Data From Teaching And Learning To Inform Teachers’ Reflective Educational Design In Inquiry-Based STEM Education

Science, Technology, Engineering and Mathematics (STEM) education is recognized as a top priority for school education worldwide and Inquiry-based teaching and learning is identified as one of the most dominant approaches. To effectively engage individual students in Inquiry tasks, appropriate guidance needs to be provided, usually by combining different digital tools such online labs, data analysis tools and modelling tools. This is a cumbersome task for teachers to perform manually since it involves (a) assessing during the education design, the type and level of tool-supported guidance to be provided to students and (b) potentially refining this level and types to meet the guidance needs of individual students based on educational data from the delivery of the educational design. Thus, in our research we target to investigate how to support this process with educational data analytics methods and tools from both the design and the delivery of educational designs, that inform teachers’ decision making for systematic reflection. To this end, the contribution of this paper is the design and evaluation of a novel “Teaching and Learning” Analytics method and supporting research prototype tool, extending the scope of purely learning analytics methods, to (a) analyze inquiry-based educational designs in terms of the tool-supported guidance they offer and (b) relate these analyses to students' educational data that are already being collected by existing learning analytics systems, so as to increase teachers’ awareness and understanding and scaffold their reflection. A two-layer evaluation methodology was adopted to evaluate both the capacity of our method to analyze educational designs in terms of appropriate guidance as well as to investigate whether the insights generated by the method offer statistically significant indicators that impact students' activity during the delivery of these educational designs. The results obtained, based on real-life educational data, argue that the proposed method and tool can support teachers to accurately analyse Inquiry-based educational designs and receive meaningful insights to improve and tailor students' learning experiences. The insights of this work aim to contribute in the research field of cognitive data analytics for teaching and learning, by investigating new ways to combine analyses of the educational design and the students’ activity, so as to inform teachers’ reflective decision making from a holistic perspective

Special issue on: Learning Analytics (Editorial)

Maria Jesus Rodriguez Triana

With the general technological advances of the recent years, current learning environments amass an abundance of data. Albeit such data offer the chance of better understand the learning process, stakeholders – learners, teachers and institutions – often need additional support to make sense of it (Dyckhoff et al., 2013; Macfadyen and Dawson, 2012). The acknowledgement of these needs is at the heart of the recent emergence of Learning Analytics (LA), a research area that draws from multiple disciplines such as educational science, information and computer science, sociology, psychology, statistics and educational data mining (Buckingham Shum and Ferguson, 2012). This multidisciplinarity in LA has motivated the work done by Ferguson (2012), which provides a first review of the drivers, development and challenges of this novel and young research area. Our understanding of learning analytics is based on the definition from the Society for Learning Analytics (SoLAR – Society for Learning Analytics1) which specifies that “Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs”. Since 2011, the Horizon reports list Learning Analytics as a hot topic in higher education and indicate the importance of data for this field (Johnson et al., 2011). Learning analytics are able to provide a fresh view on understanding of teaching and learning by observing patterns of complex data (Johnson et al., 2012). Furthermore, it will influence the evolution of higher education in a great measure. Nowadays, learners have access to a huge amount of online information having themselves the possibility of being content creators and information sharers. Therefore the quantity of available information grows in an exponential way, once that each and every citizen can access and produce information. For these purposes, learners have at their disposal many online resources, including LMSs, VLEs, MOOCs and many other online tools that facilitate the learning process and the development of competences. Taking into account these online learning facilities and therefore the learners’ acquisition of knowledge, it is also easier to measure and analyse their experiences by using learning analytics tools. Different online courses and institutions provide dashboards with information about student experiences, flaws and successes. Although the investigation of behavioural specific data makes learning analytics complex, the time comes to utilise personalised learning environments adapted to students learning paths, skills, previous knowledge, competences and motivation.

2013

Towards teacher-led design inquiry of learning

Maria Jesus Rodriguez Triana

This paper proposes “teacher-led design inquiry of learning” as a new model of educational practice and professional development. This model combines four existing models. It integrates teacher inquiry into student learning, learning design, and learning analytics, and aims to capture the essence of the synergy of these three fields. Furthermore, we identify how learning analytics and the integrated model inform each other and could help integrating learning analytics into teachers’ practice. The last claim is demonstrated through an illustrative scenario. We envision that the integration of the four models could help teachers align both the improvement of their practices and the orchestration of their classrooms. Future empirical investigation is envisaged using a design based research framework and participatory design approach to engage teachers with the integrated model in a professional development process. We envisage that the integrated model will promote quality enhancement in education at a personal and collective level, and will be used to design better learning analytics, learning design and learning enactment tools. The main limitation of the integrated model is that it requires organizational changes, and allocation of resources, in order to allow it to significantly impact practice.

2014

Introducing Productive Engagement for Social Robots Supporting Learning

Jauwairia Nasir

We have all been one such student or seen such students who can maintain the 'good student' image while playing a video game under the table or those loyal backbenchers, seemingly always distracted, who then ace their exams. These intricacies of human behaviors are just a few examples of what makes it non-trivial and challenging even for expert teachers to know how students' visible behaviors relate with learning. As research investigates ways in which robots and AI can support teachers and students, it is faced with the same challenge of inferring students' engagement; thus, making the investigation of this topic increasingly popular in educational HRI. The state of the art usually explores the relationship between the robot behaviors and the engagement state of the learner while assuming a linear relationship between engagement and learning. However, is it correct to assume that to maximize learning, one needs to maximize engagement? Furthermore, conventional supervised engagement models require human annotators to get labels. This not only is laborious but can also introduce subjectivity. Can we have machine-learning engagement models where annotations do not rely on human annotators? Additionally, with the increase in open-ended learning activities which by design employ the 'learning by failing' paradigm, in-task performance can not be the best measure for learning. Can we instead rely on multi-modal behaviors? In an effort to cater for these challenges, this thesis dives deep to identify and quantify the relationship between learning and engagement, which we term as Productive Engagement (PE). In order to develop, design, and evaluate our PE framework, (1) we first designed and developed an open-ended collaborative learning activity that served as a platform for evaluating different robot variants over time. With 98 children interacting with the baseline version from 2 international Swiss schools, we showed that in-task performance and learning are indeed not correlated. Thus, this showed the importance of not being limited to robot interventions that affect only superficial measures of students' learning. (2) Then, with learner's multi-modal behaviors, we showed that indeed there is a hidden link between learner's behaviors and learning that can be quantified, i.e., validating the proposed concept of Productive Engagement. (3) This quantifiable link surfaced three collaborative multi-modal learner profiles, by using a forward and backward clustering and classification technique, two of which are linked to higher learning. This technique gave a possibility to surface data driven labels for engagement; thus, evading the process of human annotations. We then identified similarities and differences between these learner profiles both at an aggregate and at the temporal level. (4) Based on (3), we constructed a PE score that can either be directly used as an assessment metric by a social robot in real-time or as data driven labels for building more sophisticated regression models. (5) With the learner profiles and the PE score, we designed and evaluated more advanced robot variants for the final studies with ~160 students from 7 international Swiss schools. With the design of different robot variants that employ knowledge about the learner's skills conducive to learning, rather than domain knowledge, in order to provide interventions; we provided a complementary perspective on the role of social robots in educational settings.

EPFL2022

Special issue on: Learning Analytics (Editorial)

Maria Jesus Rodriguez Triana

2013

Introducing Productive Engagement for Social Robots Supporting Learning

Jauwairia Nasir

EPFL2022