Sensory substitution | EPFL Graph Search

Sensory substitution is a change of the characteristics of one sensory modality into stimuli of another sensory modality. A sensory substitution system consists of three parts: a sensor, a coupling system, and a stimulator. The sensor records stimuli and gives them to a coupling system which interprets these signals and transmits them to a stimulator. In case the sensor obtains signals of a kind not originally available to the bearer it is a case of sensory augmentation. Sensory substitution concerns human perception and the plasticity of the human brain; and therefore, allows us to study these aspects of neuroscience more through neuroimaging. Sensory substitution systems may help people by restoring their ability to perceive certain defective sensory modality by using sensory information from a functioning sensory modality. History The idea of sensory substitution was introduced in the 1980s by Paul Bach-y-Rita as a means of using one sensory modality, mainly tactition,

Increasing Haptic Fidelity and Ergonomics in Teleoperated Surgery

Laura Santos Carreras

Over the past few decades, surgical procedures have made enormous progress, shifting from traditional open procedures to less and less invasive approaches, with the promise of smaller incisions, less complications, better cosmetic results and shorter recovery times. With these developments came a reduced dexterity and more complex control through the fulcrum effect and modified eye-hand coordination. These complications were greatly mitigated by the recent introduction of surgical robots, allowing the surgeon to sit in a comfortable posture, and restoring natural visuomotor coordination. To date, the issue of the lack of haptic feedback, which initially allowed the surgeon to intervene without vision, identify pathological tissue, feel arteries, etc., has not been resolved, despite the fact that it is crucial in certain interventions. Besides, the added value of haptic feedback is subject to controversy. Surgeons experienced in robotic surgery have adapted to the lack of haptic feedback and learned how to rely only on vision and proprioceptive cues as compensation. Nevertheless, previous studies have shown that substituting haptic information through various sensory channels can increase surgeons' performance. As a complete restoration of the sense of touch is extremely challenging in minimally invasive surgery, the advantages of haptic feedback have to be demonstrated and quantified to justify the additional cost and complexity. This is the aim of the present work. We hypothesized that tactile feedback as well as force is crucial in surgery and we investigated the haptic information involved in several representative surgical tasks. Dedicated hardware and a virtual reality environment to simulate suturing and palpation tasks were developed to address these questions. Ergonomic design guidelines were established based on an in-depth literature review and surgeons' comments gathered in a survey. These guidelines were then used to design and benchmark an ergonomic haptic handle featuring active grasping feedback and additional safety features. The results of the first two studies suggest that the benefits of haptic information highly depend on the surgical task in question. During a suturing task, force feedback significantly increased users' accuracy whereas torque feedback did not result in any significant improvement. In a palpation task, a higher recognition rate was achieved with tactile feedback than with visual sensory substitution. The performance of the haptic device integrating the ergonomic handle was assessed and compared with the standard omega.7 haptic device. Results showed that the index of performance of the original device was not degraded with the additional hardware. The index of performance was slightly increased for a manipulation task involving orientations. The ergonomic assessment of the handle showed a slight decrease of the tension in the adductor pollicis and flexor digitorium muscles, and therefore a potential decrease of fatigue. Although just a subset of surgical tasks could be investigated, the results indicate that haptic feedback and sensory substitution would greatly benefit teleoperated robotic surgery. Providing the surgeon with tactile information can potentially restore the feeling of "contact with the patient" and other surgeries that are highly reliant on the sense of touch could become possible again. This thesis presents a novel multidisciplinary approach to systematically analyzing surgical tasks in order to improve safety in two dimensions. Firstly, in the area of patient safety by providing the surgeon with haptic information to increase performance and reduce the risk of errors and secondly, in the area of surgeon safety by providing a more ergonomic master console. We expect this work to be a first step in establishing a general design method for more ergonomic surgeon consoles and trust that it will inspire research to investigate the sensory mechanisms underlying highly dexterous tasks such as those performed in surgery.

EPFL2012

Designing sensory feedback approaches for restoring touch and position feedback in upper limb amputees

Upper limb amputation disrupts most daily activities and reduces the quality of life of affected individuals. Building a suitable prosthetic limb, which can restore at least some of the lost capabilities, is a goal which has been pursued for centuries. In the last few decades, our rapidly expanding understanding of the human nervous system has unlocked impressive advances in artificial limbs. Today, commercial prosthetic hands can be controlled intuitively through voluntary muscle contractions. Nevertheless, despite leaps in the quality of modern prostheses, sensory feedback remains one of the major omissions, forcing users to rely on vision to accomplish basic tasks, such as holding a plastic cup without crushing it. Several sensory feedback strategies have recently been developed to restore tactile and proprioceptive feedback to amputees, demonstrating benefits in important areas, such as higher functional performance and increases in the sense of prosthesis ownership. Sensory feedback strategies can be distinguished based on whether the sensation they restore matches the quality (homologous feedback) or the location (somatotopic feedback) of the original sensation. Despite promising results, somatotopic tactile feedback strategies often result in unnatural sensations (e.g. electricity). Furthermore, restoration of more than a single sensory modality is rarely reported, despite being necessary to create artificial limbs capable of delivering realistic sensorimotor experiences during use. In this work, I proposed three novel and complementary strategies to improve sensory feedback restoration in upper limb prostheses. I begin by describing a non-invasive transcutaneous electrical nerve stimulation (TENS) approach aimed at restoring somatotopic tactile sensations, which is potentially applicable to all trans-radial amputees. This stimulation strategy was shown to lead to high performance during functional tasks, and compared favorably to more invasive approaches, despite a few key differences. Considering that there is no such thing as a one-size-fits-all solution for amputees, I concluded that TENS represents a viable alternative to invasive systems, especially in cases where an implant is not possible or desirable. In the second part, I proposed a sensory substitution approach to multimodal feedback, which delivered somatotopic tactile and remapped proprioceptive feedback simultaneously. This stimulation strategy relied entirely on implantable electrodes, simplifying the overall system by delivering two streams of sensory information with the same device. Using this feedback system, two amputees were able to perform interesting functional tasks, such as understanding the size and compliance of various objects, with high accuracy. Finally, I proposed a novel stimulation technique for sensory feedback designed to desynchronize induced neural activity during electrical stimulation, leading to more biomimetic patterns of activity. I discussed how this strategy could be combined with the results obtained in a recent study which I contributed to, in which we demonstrated that a model based encoding strategy resulted in more natural sensations of touch. This thesis provides evidence that advances in electrical stimulation protocols can lead to more capable prosthetic limbs. These new methods enable the delivery of multimodal, biomimetic sensory feedback and will help bridge the gap between scientific discovery and clinical translation.

EPFL2018

Wearable multimodal tactile interfaces for augmenting haptic perception

Simon Gallo

Haptic feedback has been used to faithfully restitute the haptic sense lacking during telemanipulation, but also for providing auxiliary information via the haptic channel to restore functional perception of missing or damaged senses, e.g. sensory substitution for the visually impaired. Yet, despite the growing number of applications that would benefit from haptic feedback, its penetration rate remains low. We have identified three under-researched features of haptic feedback that we believe are necessary to break the bottleneck: multimodality, thermal feedback and wearability. Indeed, while haptic perception results from the integration of multiple somatosensory inputs, the interactions between haptic modalities have rarely been investigated. Thermal feedback is an ideal candidate to explore these interactions as skin temperature is known to modulate the sensitivity of skin receptors. Finally, although haptic perception is distributed over the entire body, most multimodal haptic devices are still bulky and rigid, thus limiting their usage to specific body parts and applications. As the integration of multiple feedback modalities into a flexible display is complex, the benefits of these three features need to be demonstrated. This motivated the developments presented in this thesis. First, we miniaturized and integrated thermal feedback in multisensory platforms to investigate its role in conveying non-sensory information and modulating tactile perception. The first study showed that users are able to distinguish the temperature difference between two adjacent thermal stimuli presented under the fingertip by a high-density thermal display, highlighting the potential of miniaturized displays to deliver thermally-encoded information through small skin areas. A second study showed that modifying the skin temperature yielded changes in the precision of stiffness judgments up to 22.3%, suggesting that thermal feedback can be used to modulate the perception of complex haptic percepts. Moving towards complete wearability, we introduced a wearable multimodal display that matches the skin curvature to deliver thermo-tactile cues optimally, as supported by the high usersâ identification rates of simulated materials. Subsequently, we developed a portable hydraulic actuation that generates multimodal tactile feedback in remote 3D-printed flexible displays, paving the way to truly versatile wearable multimodal displays. Finally, a tactile suit remapping tactile and proprioceptive sensations from the virtual legs of an avatar onto the upper limbs of paraplegic patients is developed. We show that the patients were able to perceive the position of the virtual legs based on tactile feedback alone, that the use of tactile feedback in synchrony with the avatarâs walk changed the perception of the boundaries of their body, and that the modulation of the tactile stimulation induced a robust sensation of walking on different floors. These results demonstrate the potential of wearable tactile displays to create perceptual orthoses. In this work, we have developed the tools to deliver a multimodal tactile stimulation using wearable thermo-tactile displays and have shown how their unique features can impact fields such as surgical and rehabilitation robotics. These findings encourage the future development of wearable multimodal tactile displays taking advantage of the synergies between haptic modalities to improve teleoperation and sensory rehabilitation.

EPFL2016