Augmented Reality to Facilitate a Conceptual Understanding of Statics in Vocational Education

At the core of the contribution of this dissertation there is an augmented reality (AR) environment, StaticAR, that supports the process of learning the fundamentals of statics in vocational classrooms, particularly in carpentry ones. Vocational apprentices are expected to develop an intuition of these topics rather than a formal comprehension. We have explored the potentials of the AR technology for this pedagogical challenge. Furthermore, we have investigated the role of physical objects in mixed-reality systems when they are implemented as tangible user interfaces (TUIs) or when they serve as a background for the augmentation in handheld AR.

This thesis includes four studies. In the first study, we used eye-tracking methods to look for evidences of the benefits associated to TUIs in the learning context. We designed a 3D modelling task and compared users' performance when they completed it using a TUI or a GUI. The gaze measures that we analysed further confirmed the positive impact that TUIs can have on the learners' experience and enforced the empirical basis for their adoption in learning applications.

The second study evaluated whether the physical interaction with models of carpentry structures could lead to a better understanding of statics principles. Apprentices engaged in a learning activity in which they could manipulate physical models that were mechanically augmented, allowing for exploring how structures react to external loads. The analysis of apprentices' performance and their gaze behaviors highlighted the absence of clear advantages in exploring statics through manipulation. This study also showed that the manipulation might prevent students from noticing aspects relevant for solving statics problems.

From the second study we obtained guidelines to design StaticAR which implements the magic-lens metaphor: a tablet augments a small-scale structure with information about its structural behavior. The structure is only a background for the augmentation and its manipulation does not trigger any function, so in the third study we asked to what extent it was important to have it. We rephrased this question to whether users would look directly at the structure instead of seeing it only through a tablet. Our findings suggested that a shift of attention from the screen to the physical object (a structure in our case) might occur in order to sustain users' spatial orientation when they change positions. In addition, the properties of the gaze shift (e.g. duration) could depend on the features of the task (e.g. difficulty) and of the setup (e.g. stability of the augmentation).

The focus of our last study was the digital representation of the forces that act in a loaded structure. From the second study we observed that the physical manipulation failed to help apprentices understanding the way the forces interact with each other. To overcome this issue, our solution was to combine an intuitive representation (springs) with a slightly more formal one (arrows) which would show both the nature of the forces and the interaction between them. In this study apprentices used the two representations to collaboratively solve statics problems. Even though apprentices had difficulties in interpreting the two representations, there were cases in which they gained a correct intuition of statics principles from them.

In this thesis, besides describing the designed system and the studies, implications for future directions are discussed.

How Florist Apprentices Explore Bouquet Designs: Supporting Design Space Exploration for Vocational Students

Pierre Dillenbourg, Kevin Gonyop Kim, Catharine Regina Monika Maria Oertel Genannt Bierbach

Context: Exploring the design space is an important process in a design task. In this study, we considered design space exploration for the learners in vocational education and training (VET). The goal of the study was to investigate how they explore the design space while focusing on the effect of a graph-like interface on the learner’s understanding of the design space. With florists as the target profession, we investigated how the apprentices explore design variations, what they would gain from such activity, and how we can support this process. Approach: We developed a web application called BloomGraph that allows learners to explore design variations. It provides a graph-based interface that enables the systematic variation of design. Using the BloomGraph application, we conducted an experimental study with 44 florist apprentices in Switzerland to investigate the effect of the graph-based interface which provides a structured way of exploring the design space. The experimental group was given the graph-based interface to explore design variations while the control group had a linear-based interface. We compared them in terms of the number of bouquets explored, time of exploration, diversity of bouquets explored, and the learning gain in terms of the understanding of the design space measured using pre and post-tests. We also analyzed the strategies adopted by the participants for the graph navigation and the visual exploration behavior using the eye gaze data. Findings: Our analysis shows that the graph-based interface fosters a better understanding of the size of the design space and more efficient navigation towards a goal design in terms of the number of intermediate designs but with longer exploration of each intermediate design compared to the linear-based interface. Regarding the behavioral patterns in graph exploration, the participants who showed more strategic behavior in the design choices acquired a better understanding of the design space. Additionally, we trained a model that predicts the next choice of a learner using eye tracking data. It provides a reasonable accuracy that opens new possibilities for future studies. Conclusion: The findings of this study support the feasibility of design space exploration as a digital activity for VET learners and show how the learners can benefit from it. The contribution of the paper includes the validation of the idea with florist apprentices and the demonstration of how the process can be supported using a structured interface and the learner behavior analysis. This paper shows how a design exploration activity can provide an added value in the learning of an apprentice in a design-related VET system.

2021

Tangible Interfaces for Learning

Sébastien Cuendet

There have been many claims that Tangible User Interfaces (TUIs) can have a positive impact on learning. Alleged benefits include increasing usability, improving engagement and collaboration of students, and providing a better perception of the task, especially spatial ones. However, there exists little empirical data to back up these claims. Moreover, for all their potential benefits for learning, TUIs are still scarcely used in schools. This thesis explores these two issues in the specific context of vocational education and training of carpenter apprentices. The learning objectives concern spatial skills and in particular, the mapping between 2D and 3D representations. We study (1) whether TUIs can support the training of spatial skills, and if so what features allow them to do so, and (2) what kinds of classroom pedagogical scenarios TUIs can support. We follow a design-based research approach and run empirical studies, mostly in classrooms. The contributions of this thesis touch on three research domains:

Spatial skills. Our results show that TUIs can help teach spatial skills to carpenter apprentices. The tangible nature of TUIs can help the learner relate multiple representations of an object, especially for difficult problems. It can also lower the barrier to entry into a learning domain for beginners.
Learning with TUIs. According to our results, TUIs can benefit learning, but the mere fact of using TUIs does not guarantee learning. Instead, special attention needs to be given to the design of the TUI. Small design variations, such as the physical correspondence between the tangible object and its virtual representations, or the type and timing of feedback given to the user, can have a significant impact on learning.
Classroom technologies and orchestration. We explore several classroom pedagogical scenarios that TUIs can support. The most promising one is to use a TUI as part of a hybrid classroom learning activity that includes both TUI and non TUI steps. Additionally, we devise two ways to promote the integration of TUIs in classroom. First, we introduce 5 design principles that reduce the classroom orchestration load. Second, we show how new web technologies can be used to deploy TUIs in schools at a lower cost.

EPFL2013

The Complementarity of Tangible and Paper Interfaces in Tabletop Environments for Collaborative Learning

Guillaume Zufferey

The current trend in Human-Computer Interaction aims at bridging the gap between the digital and the real world, exploring novel ways to engage users with computational devices. Computers take new forms that are better integrated into our environment and can be embedded in buildings, furniture or clothes. Novel forms of interfaces take advantage of people's intuitive knowledge of everyday objects to offer more direct and natural interactions. Tangible User Interfaces (TUIs) allow users to interact with digital objects through tangible artifacts, building on their rich physical affordances. Paper User Interfaces (PUIs) add digital capabilities to paper documents, synchronizing for instance their content with their digital counterpart. Unique properties of paper are also used to create engaging and intuitive interfaces to computer applications. This dissertation is interested in the complementarity of tangible and paper interfaces in tabletop environments. We introduce the concept of Tangible and Paper Environments (TaPEs) where Interactive Paper Forms (IPFs), a particular type of PUIs based on the paper form metaphor, are used as a complementary interface to a TUI. We evaluate the potential of IPFs to overcome two main shortcomings of TUIs, in terms of scalability and pedagogy. The scalability issue comes from the limited expressiveness of task-specific physical artifacts, which offer rich physical affordances but limit the complexity of applications that can be controlled by a TUI. The pedagogy issue is raised by the lack of consistent evidence regarding the use of physical manipulatives in educational settings, which is one of the main application domain of TUIs. IPFs overcome the scalability issue by offering a set of generic interaction elements that allow TaPEs to cope with applications of any complexity. In a pedagogical setting, IPFs present learners with abstract representation which facilitate understanding by the embodied and concrete representations offered by tangible artifacts. A TaPE, the Tinker Environment, has been developed with two logistics teachers in the context of the Swiss vocational training system. It consists of a warehouse physical small-scale model (TUI) and TinkerSheets, our implementation of IPFs. It aims at helping apprentices understand theoretical concepts presented at schools. We followed a Design-based Research (DBR) approach: ten studies were conducted during the development of the Tinker Environment in authentic classroom settings. Controlled experiments were conducted to address specific questions. v The general research questions concern the respective affordances of paper and tangible components of TaPEs. The analysis is not limited to usability aspects but also considers their impact on group problem-solving activities and their potential in terms of integration of the system in its context of use. A descriptive model is proposed, built around three interaction circles: individual (usability), group (collaboration) and context (integration). Results identify design guidelines that limit the impact of the less direct interaction modality offered by IPFs, allowing TaPEs to overcome the scalability issue while supporting rich interactions. At the group level, observations of groups of apprentices solving problems around the Tinker Environment show that the consistent physical interaction modality offered by TaPEs naturally supports collaborative interactions. Apprentices tend to take implicit roles based on their location around the system. Regarding the context circle, we observed that carefully designed IPFs play the role of bridges between offline and online activities and contribute to a tight integration of the system in a its context (i.e. a classroom). The specific research questions address the potential of the Tinker Environment in this pedagogical context and its appropriation by teachers. The observations conducted with the Tinker Environment show that the warehouse small-scale model reduces the complexity of problems and allows apprentices to engage in meaningful problem-solving activities. Controlled experiments comparing a TUI to a mulitouch interface demonstrate that tangible artifacts lead to a higher learning gain and an increased performance in a problem-solving activity. Collaboration quality and perceived playfulness are also improved. The teacher plays a central role in the use of the environment, guiding apprentices through activities and encouraging reflections during debriefing sessions. The design of IPFs, emphasizing either their interface or document nature, has a strong influence on their ability to support teachers. We finally discuss the two-way adaptation process that took place between teachers and the system during the development of the Tinker Environment.

EPFL2010