Tangible interfaces: when physical-virtual coupling may be detrimental to learning

Tangible user interfaces (TUIs) have been the focus of much attention recently in the HCI and learning communities. Although TUIs seem to intuitively offer potential to increase the learning experience, there have been questions about whether they actually impact learning positively. TUIs offer new ways of interactions and it is essential to understand how the design choices made for these new interactions affect learning. One element that is key in the learning process is how and when feedback is provided. In this article, we focus on the effect of co-located immediate process-level feedback on learning. We report the results of a study in which 56 participants used a TUI to complete tasks related to the training of spatial skills. Half of the students accomplished the tasks with immediate and co-located feedback from the system, while the other half of the students did not receive any feedback. Results show that participants who did not receive feedback manipulated less, reflected more, and in the end learned more than those who received feedback.

Augmented Reality to Facilitate a Conceptual Understanding of Statics in Vocational Education

Lorenzo Lucignano

At the core of the contribution of this dissertation there is an augmented reality (AR) environment, StaticAR, that supports the process of learning the fundamentals of statics in vocational classrooms, particularly in carpentry ones. Vocational apprentices are expected to develop an intuition of these topics rather than a formal comprehension. We have explored the potentials of the AR technology for this pedagogical challenge. Furthermore, we have investigated the role of physical objects in mixed-reality systems when they are implemented as tangible user interfaces (TUIs) or when they serve as a background for the augmentation in handheld AR. This thesis includes four studies. In the first study, we used eye-tracking methods to look for evidences of the benefits associated to TUIs in the learning context. We designed a 3D modelling task and compared users' performance when they completed it using a TUI or a GUI. The gaze measures that we analysed further confirmed the positive impact that TUIs can have on the learners' experience and enforced the empirical basis for their adoption in learning applications. The second study evaluated whether the physical interaction with models of carpentry structures could lead to a better understanding of statics principles. Apprentices engaged in a learning activity in which they could manipulate physical models that were mechanically augmented, allowing for exploring how structures react to external loads. The analysis of apprentices' performance and their gaze behaviors highlighted the absence of clear advantages in exploring statics through manipulation. This study also showed that the manipulation might prevent students from noticing aspects relevant for solving statics problems. From the second study we obtained guidelines to design StaticAR which implements the magic-lens metaphor: a tablet augments a small-scale structure with information about its structural behavior. The structure is only a background for the augmentation and its manipulation does not trigger any function, so in the third study we asked to what extent it was important to have it. We rephrased this question to whether users would look directly at the structure instead of seeing it only through a tablet. Our findings suggested that a shift of attention from the screen to the physical object (a structure in our case) might occur in order to sustain users' spatial orientation when they change positions. In addition, the properties of the gaze shift (e.g. duration) could depend on the features of the task (e.g. difficulty) and of the setup (e.g. stability of the augmentation). The focus of our last study was the digital representation of the forces that act in a loaded structure. From the second study we observed that the physical manipulation failed to help apprentices understanding the way the forces interact with each other. To overcome this issue, our solution was to combine an intuitive representation (springs) with a slightly more formal one (arrows) which would show both the nature of the forces and the interaction between them. In this study apprentices used the two representations to collaboratively solve statics problems. Even though apprentices had difficulties in interpreting the two representations, there were cases in which they gained a correct intuition of statics principles from them. In this thesis, besides describing the designed system and the studies, implications for future directions are discussed.

EPFL2018

Tangible vs. Virtual Representations: when Tangibles Benefit the Training of Spatial Skills

Engin Walter Bumbacher, Sébastien Cuendet, Pierre Dillenbourg

Tangible user interfaces (TUIs) have been the focus of much attention in the HCI and learning communities because of their many potential benefits for learning. However, there have recently been debates about whether TUIs can actually increase learning outcomes and if so, under which conditions. In this article, we investigate the effect of object representation (physical vs. virtual) on learning in the domain of spatial skills. We ran a comparative study with 46 participants to measure the effects of the object representation on the ability to establish a link between 2D and 3D representations of an object. The participants were split into two conditions: in the first one, the 3D representation of the object was virtual; in the second one, it was tangible. Findings show that in both conditions the TUI led to a significant improvement of the spatial skills. The learning outcomes were not different between the two conditions, but the performance during the activities was significantly higher when using the tangible representation as opposed to the virtual one, and even more so in for difficult cases.

2012

Tangible Interfaces for Learning

Sébastien Cuendet

There have been many claims that Tangible User Interfaces (TUIs) can have a positive impact on learning. Alleged benefits include increasing usability, improving engagement and collaboration of students, and providing a better perception of the task, especially spatial ones. However, there exists little empirical data to back up these claims. Moreover, for all their potential benefits for learning, TUIs are still scarcely used in schools. This thesis explores these two issues in the specific context of vocational education and training of carpenter apprentices. The learning objectives concern spatial skills and in particular, the mapping between 2D and 3D representations. We study (1) whether TUIs can support the training of spatial skills, and if so what features allow them to do so, and (2) what kinds of classroom pedagogical scenarios TUIs can support. We follow a design-based research approach and run empirical studies, mostly in classrooms. The contributions of this thesis touch on three research domains:

Spatial skills. Our results show that TUIs can help teach spatial skills to carpenter apprentices. The tangible nature of TUIs can help the learner relate multiple representations of an object, especially for difficult problems. It can also lower the barrier to entry into a learning domain for beginners.
Learning with TUIs. According to our results, TUIs can benefit learning, but the mere fact of using TUIs does not guarantee learning. Instead, special attention needs to be given to the design of the TUI. Small design variations, such as the physical correspondence between the tangible object and its virtual representations, or the type and timing of feedback given to the user, can have a significant impact on learning.
Classroom technologies and orchestration. We explore several classroom pedagogical scenarios that TUIs can support. The most promising one is to use a TUI as part of a hybrid classroom learning activity that includes both TUI and non TUI steps. Additionally, we devise two ways to promote the integration of TUIs in classroom. First, we introduce 5 design principles that reduce the classroom orchestration load. Second, we show how new web technologies can be used to deploy TUIs in schools at a lower cost.

EPFL2013