Those are Your Legs: The Effect of Visuo-Spatial Viewpoint on Visuo-Tactile Integration and Body Ownership

Experiencing a body part as one's own, i.e., body ownership, depends on the integration of multisensory bodily signals (including visual, tactile, and proprioceptive information) with the visual top down signals from peripersonal space. Although it has been shown that the visuo-spatial viewpoint from where the body is seen is an important visual top-down factor for body ownership, different studies have reported diverging results. Furthermore, the role of visuo-spatial viewpoint (sometime also called first person perspective) has only been studied for hands or the whole body, but not for the lower limbs. We thus investigated whether and how leg visuo-tactile integration and leg ownership depended on the visuo-spatial viewpoint from which the legs were seen and the anatomical similarity of the visual leg stimuli. Using a virtual leg illusion, we tested the strength of visuo-tactile integration of leg stimuli using the crossmodal congruency effect (CCE) as well as the subjective sense of leg ownership (assessed by a questionnaire). Fifteen participants viewed virtual legs or non-corporeal control objects, presented either from their habitual first person viewpoint or from a viewpoint that was rotated by 900 (third-person viewpoint), while applying visuo-tactile stroking between the participants legs and the virtual legs shown on a head-mounted display. The data show that the firstperson visuo-spatial viewpoint significantly boosts the visuo-tactile integration as well as the sense of leg ownership. Moreover, the viewpoint dependent increment of the visuotactile integration was only found in the conditions when participants viewed the virtual legs (absent for control objects). These results confirm the importance of first person visuo-spatial viewpoint for the integration of visuo-tactile stimuli and extend findings from the upper extremity and the trunk to visuo-tactile integration and ownership for the legs.

Full body action remapping of peripersonal space: The case of walking

Olaf Blanke, Petr Grivaz, Hervé Lissek, Patrick Marmaroli, Jean-Paul Noel, Andrea Serino

The space immediately surrounding the body, i.e. peripersonal space (PPS), is represented by populations of multisensory neurons, from a network of premotor and parietal areas, which integrate tactile stimuli from the body’s surface with visual or auditory stimuli presented within a limited distance from the body. Here we show that PPS boundaries extend while walking. We used an audio–tactile interaction task to identify the location in space where looming sounds affect reaction time to tactile stimuli on the chest, taken as a proxy of the PPS boundary. The task was administered while participants either stood still or walked on a treadmill. In addition, in two separate experiments, subjects either received or not additional visual inputs, i.e. optic flow, implying a translation congruent with the direction of their walking. Results revealed that when participants were standing still, sounds boosted tactile processing when located within 65–100 cm from the participants’ body, but not at farther distances. Instead, when participants were walking PPS expands as reflected in boosted tactile processing at ~1.66 m. This was found despite the fact the spatial relationship between the participant’s body and the sound’s source did not vary between the Standing and the Walking condition. This expansion effect on PPS boundaries due to walking was the same with or without optic flow, suggesting that kinematics and proprioceptive cues, rather than visual cues, are critical in triggering the effect. These results are the first to demonstrate an adaptation of the chest’s PPS representation due to whole body motion and are compatible with the view that PPS constitutes a dynamic sensory–motor interface between the individual and the environment.

Pergamon-Elsevier Science Ltd2015

Wearable multimodal tactile interfaces for augmenting haptic perception

Simon Gallo

Haptic feedback has been used to faithfully restitute the haptic sense lacking during telemanipulation, but also for providing auxiliary information via the haptic channel to restore functional perception of missing or damaged senses, e.g. sensory substitution for the visually impaired. Yet, despite the growing number of applications that would benefit from haptic feedback, its penetration rate remains low. We have identified three under-researched features of haptic feedback that we believe are necessary to break the bottleneck: multimodality, thermal feedback and wearability. Indeed, while haptic perception results from the integration of multiple somatosensory inputs, the interactions between haptic modalities have rarely been investigated. Thermal feedback is an ideal candidate to explore these interactions as skin temperature is known to modulate the sensitivity of skin receptors. Finally, although haptic perception is distributed over the entire body, most multimodal haptic devices are still bulky and rigid, thus limiting their usage to specific body parts and applications. As the integration of multiple feedback modalities into a flexible display is complex, the benefits of these three features need to be demonstrated. This motivated the developments presented in this thesis. First, we miniaturized and integrated thermal feedback in multisensory platforms to investigate its role in conveying non-sensory information and modulating tactile perception. The first study showed that users are able to distinguish the temperature difference between two adjacent thermal stimuli presented under the fingertip by a high-density thermal display, highlighting the potential of miniaturized displays to deliver thermally-encoded information through small skin areas. A second study showed that modifying the skin temperature yielded changes in the precision of stiffness judgments up to 22.3%, suggesting that thermal feedback can be used to modulate the perception of complex haptic percepts. Moving towards complete wearability, we introduced a wearable multimodal display that matches the skin curvature to deliver thermo-tactile cues optimally, as supported by the high usersâ identification rates of simulated materials. Subsequently, we developed a portable hydraulic actuation that generates multimodal tactile feedback in remote 3D-printed flexible displays, paving the way to truly versatile wearable multimodal displays. Finally, a tactile suit remapping tactile and proprioceptive sensations from the virtual legs of an avatar onto the upper limbs of paraplegic patients is developed. We show that the patients were able to perceive the position of the virtual legs based on tactile feedback alone, that the use of tactile feedback in synchrony with the avatarâs walk changed the perception of the boundaries of their body, and that the modulation of the tactile stimulation induced a robust sensation of walking on different floors. These results demonstrate the potential of wearable tactile displays to create perceptual orthoses. In this work, we have developed the tools to deliver a multimodal tactile stimulation using wearable thermo-tactile displays and have shown how their unique features can impact fields such as surgical and rehabilitation robotics. These findings encourage the future development of wearable multimodal tactile displays taking advantage of the synergies between haptic modalities to improve teleoperation and sensory rehabilitation.

EPFL2016

Self-grounded vision: hand ownership modulates visual location through cortical beta and gamma oscillations

Javier Bello Ruiz, Fosco Bernasconi, Olaf Blanke, Herberto Dhanis Pedro De Barros Camacho, Nathan Quentin Faivre, Roy Salomon, Michele Scandola

Vision is known to be shaped by context, defined by environmental and bodily signals. In the Taylor illusion, the size of an afterimage projected on one's hand changes according to proprioceptive signals conveying hand position. Here, we assessed whether the Taylor illusion does not just depend on the physical hand position, but also on bodily self-consciousness as quantified through illusory hand ownership. Relying on the somatic rubber hand illusion, we manipulated hand ownership, such that participants embodied a rubber hand placed next to their own hand. We found that an afterimage projected on the participant's hand drifted depending on illusory ownership between the participants' two hands, showing an implication of self-representation during the Taylor illusion. Oscillatory power analysis of electroencephalographic signals showed that illusory hand ownership was stronger in participants with stronger alpha suppression over left sensorimotor cortex, whereas the Taylor illusion correlated with higher beta/gamma power over frontotemporal regions. Higher gamma connectivity between left sensorimotor and inferior parietal cortex was also found during illusory hand ownership. These data show that afterimage drifts in the Taylor illusion do not only depend on the physical hand position but also on subjective ownership, which itself is based on the synchrony of somatosensory signals from the two hands. The effect of ownership on afterimage drifts is associated with beta/gamma power and gamma connectivity between frontoparietal regions and the visual cortex. Together, our results suggest that visual percepts are not only influenced by bodily context but are self-grounded, mapped on a self-referential frame.

2017

Wearable multimodal tactile interfaces for augmenting haptic perception

Simon Gallo

EPFL2016