Barbara Bruno

Related publications (41)

A Research-Practice Partnership to Introduce Computer Science in Secondary School: Lessons from a Pilot Program

Barbara Bruno, Helena Kovacs, Francesco Mondada, Jean-Philippe Pellet

Context: Introducing Computer Science (CS) into formal education can be challenging, notably when considering the numerous stakeholders involved which include the students, teachers, schools, and policy makers. We believe these perspectives should be considered conjointly, which is possible within Research Practice Partnerships (RPPs). RPPs look to bridge research-practice gaps and have seen an increase in the field of education and CS-education. Unfortunately, RPPs are considered to be under-researched, in addition to presenting their own challenges. Objectives: To the purpose of assessing how RPPs may support the successful introduction of CS into formal education, we investigate three perspectives (students, teachers, and RPP stakeholders) and their interplay within the context of a multi-institution RPP conducting a pilot program to introduce CS to secondary school students. Methods: A mixed methods analysis was employed to triangulate data in a concurrent triangulation design. The data included i) 3 surveys distributed over the semester to 106 grade 9 students (ages 12-14), ii) four teacher-journals, iii) 2 interviews and 4 focus groups with the teachers and representatives of the partner institutions. Findings: From the students’ perspective, while their self-efficacy increased, their motivation decreased throughout the semester due to a miss-match between their expectations and the course. The findings also indicate that gender biases and heterogeneity are already present in grade 9. From the teachers’ perspective, co-constructing the study plan, having access to regular support and collaborating within a community of practice when starting to teach CS all facilitated the teachers’ experience. Finally, from the RPP’s perspective the collaboration between stakeholders and having researchers evaluate the program were considered to be key elements in the pilot program. However, there appears to be a research-practice gap, in big part due to limited interactions between researchers and curriculum designers, and researchers and the teachers in the field. Conclusions: From the students’ perspective it appears relevant to introduce CS i) prior to secondary school to address motivation and bias-related issues early on, and ii) to all students to avoid participation being motivation-, stereotype-, or belief-driven, and risk broadening the gap between students, iii) all the while being attentive to course format and content to ensure that the course meets students’ expectations and fosters autonomous motivation. From the teachers’ perspective, while the provided support met the teachers’ needs, it is essential to find means of scaling such approaches when looking to deploy CS-curricular reforms to entire administrative regions. Finally, from the RPP’s perspective i) teachers’ should be given a voice in the RPP to better align with the field, and ii) researchers’ roles should be reconsidered to move beyond being only evaluators, and towards having a more co-constructive role in setting up the curricular reform. Recommendations are provided for researchers and practitioners involved in CS curricular reforms.

2023

Evaluation of teachers’ orchestration tools usage in robotic classrooms

Barbara Bruno, Pierre Dillenbourg, Aditi Kothiyal, Sina Shahmoradi

Teachers’ self-efficacy in managing classrooms is an important consideration when it comes to bringing educational robots to classrooms. Orchestration tools support teachers by providing awareness indicators of students’ progress as well as levers to control the flow of the lesson. We designed and evaluated the impact of an orchestration tool for a series of robot-based learning activities to teach a basic concept in mathematics to children, aged 7-10. Six teachers in primary schools across Switzerland used the orchestration tool to manage the activities in six sessions involving a total of ninety-one students. We observed teachers’ usage of the orchestration tool during the sessions and interviewed them after the sessions about the usefulness of these functionalities. Our findings show that even though teachers used the tool in different ways from each other, in general, it supported them in their classroom orchestration practices, mainly to manage the activity sequence and get aware of the robot technical failures and, to a lesser degree, get aware of students’ progress for the purpose of activity transitions and enriching class discussions. We discuss the theoretical implications of these results, relating our findings to the literature on classroom orchestration tool design, especially highlighting the importance of educational level and the type of learning technology as contextual factors affecting teachers’ usage of orchestration tools. We also provide implications for designing orchestration tools, focusing on the necessity of providing different types of awareness indicators and multiple options for activity management to fulfil the variety of teachers’ orchestration needs.

2023

Modelling the sustainability of a primary school digital education curricular reform and professional development program

Barbara Bruno, Francesco Mondada, Emilie-Charlotte Monnier

Sustaining changes in teachers’ practices is a challenge that determines the success of curricular reforms, from which Digital Education (DE) is not exempt. As the literature on sustainability is considered “scarce” and “scattered”, long-term studies modelling the factors impacting teachers’ sustained uptake of DE pedagogical content are lacking. Thus, we investigate whether and how 287 in-service teachers sustained a primary school DE curricular reform over a year after they completed their two-year DE professional development program. We model the sustainability of the reform through Structural Equation Modelling, and identify critical sustainability-factors. The validated Sustainable Adoption of Digital Education (SADE) model confirms that sustainability in the fourth year of the reform depends on perceived usefulness of teaching the new content, ease of implementation, and access to sufficient support in schools. Such factors should thus be evaluated, accounted for in the implementation phase of the reform, and sustained over time. The findings confirm that the DE curricular reform model contributes to positive self-efficacy to teach DE, provides sufficient in-school support, and promotes increasing adoption over time. However, as teachers’ practices have not yet stabilised, and teachers may still adopt more to cover the breadth of DE-concepts, it is important to remain attentive to remaining sustainability barriers: lack of time, effort required to teach DE with teachers preferring to delegate, and lack of student learning evidence, the latter being a significant challenge to address in the literature. These barriers must therefore be jointly addressed by researchers and practitioners in the field in order to promote the sustainability of the reform.

2023

, ,

With the introduction of Computer Science (CS) into curricula worldwide, researchers have investigated whether CS could be introduced transversally, as a support other disciplines. Few however consider both student learning and the teachers’ perspective in their assessments. In a co-constructive approach to translational research, we collaborated with teachers, in two case studies involving two classes each, to investigate how CS content could be used transversally. More specifically, teacher inputs and student learning data were combined in a mixed methods analysis to determine whether two CS Unplugged (CSU) activities from the curriculum could be leveraged to teach disciplinary content in primary school. The findings indicated that the CSU activities could be leveraged for maths and spelling, but require validation at a larger scale. More important, though, are the takeaways of the co-constructive experience with teachers. Interestingly the benefits of co-construction went both ways. While researchers gained a deeper understanding of the effectiveness of the activities, teachers i) appreciated having detailed student learning analyses, devoid of biases and impressions, ii) changed their perspective about researchers, and iii) were open to other collaborative research initiatives which they found to be drivers of change and innovation in their practice. The way co-construction was approached also impacted the teachers’ perception of their experience. Indeed, while one study was researcher-driven and included teachers, the other was teacher-driven and included researchers. When teacher-driven, the teachers felt they had a more active role, thus reflecting more on the study design, results and implications. Provided the positive impact that co-construction may have on innovation in teacher practices, different co-construction modalities must be investigated, as well as their implications on all stakeholders, including researchers.

ACM2022

A game-based approach for evaluating and customizing handwriting training using an autonomous social robot

Barbara Bruno, Daniel Carnieto Tozadore, Pierre Dillenbourg, Chenyang Wang

Handwriting learning is a long and complex process that takes about ten years to be fully mastered. Nearly one-third of all children aged 4-12 experiences handwriting difficulties and, sadly, most of them are left to fight them on their own, due to the scarcity of tools for the detection and remediation of such difficulties. Building on state-of-the-art digital solutions for automated handwriting assessment and the training of specific handwriting-related skills, in this article we discuss requirements, rationale, and architecture of a system for handwriting training, which relies on a social robot as a mediator agent, offering personalized training and suggestions. The system is envisioned to operate autonomously and to support long-term interactions via personalization. Preliminary validation of the system in an experiment with 31 children showed its potential not only for autonomously guiding handwriting training sessions but also for its inclusion in the teachers' practice.

2022

A Speech-based Productive Engagement Metric for Real-time Human-Robot Interaction in Collaborative Educational Contexts

Barbara Bruno, Pierre Dillenbourg, Jauwairia Nasir

One of the major challenges in educational contexts is to correctly interpret the student's learning process in real-time, which is a necessary pre-requisite for timely and appropriate interventions. In educational HRI, multiple solutions have been proposed to endow robots with this key ability, typically relying on observable proxies such as in-task performance, affective behavior, engagement with the robot, etc. In this methodology paper, we propose and validate a metric for the real-time analysis of the behaviour of learners, allowing to assess whether they are engaged in meaningful learning behaviours. Specifically, building on the previously proposed concept of Productive Engagement, that inherently links learning with engagement, we hereby propose methods to quantify and compute it reliably in real-time. The training and testing of the methods is done using the open access PE-HRI-temporal dataset, that provides team level multi-modal temporal behavioral data, built from a study done with 68 students (34 teams) in a collaborative human-human educational activity mediated by a robot.

2022

Cellulan World: Interactive Platform to Learn Swarm Behaviors

Barbara Bruno, Pierre Dillenbourg, Hala Khodr, Aditi Kothiyal

Swarming behaviors are ubiquitous in nature, and an important topic to teach and learn. In the learning sciences domain of complex systems understanding, studies have shown that novices tend to assume that centralized control and leadership should exist whenever they see complex patterns. In this paper, we present an educational framework consisting of a swarm of virtual and/or physical agents (robots) as a game-based learning environment to introduce the underlying rules of swarm behaviors.

IFAAMAS2022

Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt)

Barbara Bruno

Introduction: With the increasing amount of research around Computational Thinking (CT) and endeavors introducing CT into curricula worldwide, assessing CT at all levels of formal education is of utmost importance to ensure that CT-related learning objectives are met. This has contributed to a progressive increase in the number of validated and reliable CT assessments for K-12, including primary school. Researchers and practitioners are thus required to choose among multiple instruments, often overlapping in their age validity. Methods: In this study, we compare the psychometric properties of two of these instruments: the Beginners' CT test (BCTt), developed for grades 1–6, and the competent CT test (cCTt), validated for grades 3–4. Classical Test Theory and Item Response Theory (IRT) were employed on data acquired from 575 students in grades 3–4 to compare the properties of the two instruments and refine the limits of their validity. Results: The findings (i) establish the detailed psychometric properties of the BCTt in grades 3–4 for the first time, and (ii) through a comparison with students from the same country, indicate that the cCTt should be preferred for grades 3–4 as the cCTt is able to discriminate between students of low and medium ability. Conversely, while the BCTt, which is easier, shows a ceiling effect, it is better suited to discriminate between students in the low ability range. For these grades, the BCTt can thus be employed as a screening mechanism to identify low ability students. Discussion: In addition to providing recomendations for use of these instruments, the findings highlight the importance of comparing the psychometric properties of existing assessments, so that researchers and practitioners, including teachers and policy makers involved in digital education curricular reforms, may take informed decisions when selecting assessments.

2022

This dataset contains quantitative student data acquired during the administration of two validated Computational Thinking (CT) assessments for upper primary school (grades 3 and 4): the Beginners' CT test (BCTt) [1] and the comptent CT test (cCTt) [2] To compare the psychometric properties of both instruments a comparative analysis was conducted with data acquired in schools in Portugal from the same school districts. More specifically, we analyse the results of: - the BCTt test administered in March 2020 to 374 students in grades 3-4, - the cCTt test administered in April 2021 to 201 different students in grades 3-4. These students had no prior experience in Computational Thinking, as this was not part of the national curriculum at the times of administration. The detailed psychometric comparison is published in Frontiers in Psychology - Educational Psychology [3] and provides indications regarding the use of both instruments for grades 3-4. A README is included and provides additional information regarding : - the requirements for re-use. - the specific content of the 2 csv files The BCTt is available upon request to maria.zapata@urjc.es and the cCTt items are available in [2] with an editable version being available upon request to laila.elhamamsy@epfl.ch. In case of other inquiries, please contact: laila.elhamamsy@epfl.ch, maria.zapata@urjc.es or pedro.marcelino@treetree2.org References [1] M. Zapata-Cáceres, E. Martín-Barroso and M. Román-González, "Computational Thinking Test for Beginners: Design and Content Validation," 2020 IEEE Global Engineering Education Conference (EDUCON), 2020, pp. 1905-1914, doi: 10.1109/EDUCON45650.2020.9125368. [2] El-Hamamsy, L., Zapata-Cáceres, M., Barroso, E. M., Mondada, F., Zufferey, J. D., & Bruno, B. (2022). The Competent Computational Thinking Test: Development and Validation of an Unplugged Computational Thinking Test for Upper Primary School. Journal of Educational Computing Research, 60(7), 1818–1866. https://doi.org/10.1177/07356331221081753 [3] Laila El-Hamamsy* , María Zapata-Cáceres, Pedro Marcelino, Jessica Dehler Zufferey, Barbara Bruno, Estefanía Martín-Barroso and Marcos Román-González (2022). Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: the Beginners' CT test (BCTt) and the competent CT test (cCTt). Front. Psychol. doi:10.3389/fpsyg.2022.1082659 The NCCR Robotics, Madrid Regional Government (project e-Madrid-CM, P2018/TCS-4307), Structural Funds (FSE and FEDER), and Fundação Calouste Gulbenkian (Gulbenkian Programme for Knowledge) funded this project {"references": ["M. Zapata-C\u00e1ceres, E. Mart\u00edn-Barroso and M. Rom\u00e1n-Gonz\u00e1lez, "Computational Thinking Test for Beginners: Design and Content Validation,"\u00a02020 IEEE Global Engineering Education Conference (EDUCON), 2020, pp. 1905-1914, doi: 10.1109/EDUCON45650.2020.9125368.", "El-Hamamsy, L., Zapata-C\u00e1ceres, M., Barroso, E. M., Mondada, F., Zufferey, J. D., & Bruno, B. (2022). The Competent Computational Thinking Test: Development and Validation of an Unplugged Computational Thinking Test for Upper Primary School. Journal of Educational Computing Research, 60(7), 1818\u20131866. https://doi.org/10.1177/07356331221081753", "Laila El-Hamamsy* ,\u00a0Mar\u00eda Zapata-C\u00e1ceres,\u00a0Pedro Marcelino,\u00a0Jessica Dehler Zufferey,\u00a0Barbara Bruno,\u00a0Estefan\u00eda Mart\u00edn-Barroso\u00a0and\u00a0Marcos Rom\u00e1n-Gonz\u00e1lez\u00a0(2022). Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: the Beginners' CT test (BCTt) and the competent CT test (cCTt).\u00a0Front. Psychol.\u00a0doi:10.3389/fpsyg.2022.1082659"]}

2022

This dataset contains the quantitative teacher data used to analyse an in service teacher training program for Computer Science that took place from September 2019 to March 2020 in the Canton Vaud in Switzerland. Approximately 180 teachers from the the 5th and 6th grade in primary school (ages 9-11) participated in 3 days of training sessions. At the end of each training session, teachers were asked to fill in a web-based questionnaire providing information relating to their perception of the training sessions and adoption of the computer science activities. The surveys were analysed from three perspectives which are detailed in the corresponding article (the professional development program's perspective, the activities' perspective, the teacher's perspective). The present repository thus contains three csv files, one per analysis. A README is included and provides additional information regarding : - the requirements for re-use. - the survey instrument used - the specific content of the 3 csv files

2022