Active tactile exploration using a brain-machine-brain interface

Brain–machine interfaces1, 2 use neuronal activity recorded from the brain to establish direct communication with external actuators, such as prosthetic arms. It is hoped that brain–machine interfaces can be used to restore the normal sensorimotor functions of the limbs, but so far they have lacked tactile sensation. Here we report the operation of a brain–machine–brain interface (BMBI) that both controls the exploratory reaching movements of an actuator and allows signalling of artificial tactile feedback through intracortical microstimulation (ICMS) of the primary somatosensory cortex. Monkeys performed an active exploration task in which an actuator (a computer cursor or a virtual-reality arm) was moved using a BMBI that derived motor commands from neuronal ensemble activity recorded in the primary motor cortex. ICMS feedback occurred whenever the actuator touched virtual objects. Temporal patterns of ICMS encoded the artificial tactile properties of each object. Neuronal recordings and ICMS epochs were temporally multiplexed to avoid interference. Two monkeys operated this BMBI to search for and distinguish one of three visually identical objects, using the virtual-reality arm to identify the unique artificial texture associated with each. These results suggest that clinical motor neuroprostheses might benefit from the addition of ICMS feedback to generate artificial somatic perceptions associated with mechanical, robotic or even virtual prostheses.

Integration of a virtual reality based arm in primary somatosensory cortex

Hannes Bleuler, Miguel Nicolelis, Solaiman Shokur

Abstract: Recent advances in brain-machine interfaces (BMIs) have demonstrated the possibility of motor neuroprosthetics directly controlled by brain activity. Ideally neuroprosthetic limbs should be integrated in the body schema of the subject. To explore the ways to enhance such incorporation, we recorded modulations of neuronal ensemble activity in the primary somatosensory (S1) cortex during tactile stimulation simulated in virtual reality (VR) under conditions known to evoke a rubber-hand illusion. A realistic 3D mesh represented monkey body in VR. The monkey’s arms were hidden by an opaque plate and virtual arms projected on the plate. A robotic brush, also hidden from the monkey, touched various locations on forearms of the monkey and was synchronized with a virtual brush touching the projected VR arms. Additionally, we implemented tactile stimulation with air puffs. We have tested various combination of tactile (physical touch), visual (VR arm being touched) and sound (robotic brush touching the arm) inputs: synchronous tactile and visual (T-VR), tactile without visual (T), and visual only (VR). Neuronal ensemble activity was recorded from S1 and primary motor cortex (M1). We found differences in both S1 and M1 activities across the stimulation types. In particular S1 responses to T-VR were stronger than for T. Moreover, S1 neurons were modulated during visual stimulation without touch (VR), suggesting S1 activation as a neuronal mechanism of the rubber-hand illusion. Further, we decoded stimulation parameters from the activity of large neuronal populations. These results suggest a flexible and distributed representation of somatosensory information in the cortex, which can be modified by visual feedback from the body and/or artificial actuators.

2010

Bimanual brain-machine interface

Zheng Li, Miguel Nicolelis, Solaiman Shokur

Brain-machine interfaces (BMIs) - devices that connect brain areas to external actuators - strive to restore limb mobility and sensation to patients suffering from paralysis or limb loss. Here we report a novel BMI that controls two virtual arms simultaneously. The development of BMIs for bimanual control is important because even the most basic daily movements such as opening a jar or buttoning a shirt require two arms. We for the first time have designed and implemented a bimanual BMI where activity of multiple cortical areas is translated in real-time into center-out reaching movements performed by two virtual arms. Eight multielectrode arrays, a total of 768 electrode channels, were implanted in the primary motor (M1), sensory (S1), supplementary motor (SMA), dorsal premotor (PMd), and posterior parietal (PP) cortices of both hemispheres of a rhesus monkey. Movement kinematics of each arm were extracted from the same ensemble of 400 neurons using a Wiener filter and an unscented Kalman filter (UKF). Typically, a single neuron contributed to the movements of both left and right arms. Movements were enacted by arms of a virtual rhesus monkey avatar on a computer screen presented in first-person to the monkey. On each trial, the virtual arms moved their central locations to peripheral targets presented simultaneously on the right and left sides of the computer screen. Peri-event time histograms and linear discriminant analysis revealed a highly distributed encoding scheme, with movement directions of both limbs represented by both ipsilateral and contralateral areas. Furthermore, movements were represented by multiple cortical regions, including both primary and non-primary motor areas which have been previously identified areas important for bimanual coordination. Over the course of several weeks of real-time BMI control, the monkey’s performance clearly improved both when the monkey continued to move the joystick and when the joystick was removed. These results support the feasibility of cortically-driven clinical neural prosthetics for bimanual operations.

2012

Decoding bipedal locomotion from the rat sensorimotor cortex

Grégoire Courtine, Lucia Florinda Friedli Wittler, Silvestro Micera, Jacopo Rigosa, Rubia van den Brand

Objective. Decoding forelimb movements from the firing activity of cortical neurons has been interfaced with robotic and prosthetic systems to replace lost upper limb functions in humans. Despite the potential of this approach to improve locomotion and facilitate gait rehabilitation, decoding lower limb movement from the motor cortex has received comparatively little attention. Here, we performed experiments to identify the type and amount of information that can be decoded from neuronal ensemble activity in the hindlimb area of the rat motor cortex during bipedal locomotor tasks. Approach. Rats were trained to stand, step on a treadmill, walk overground and climb staircases in a bipedal posture. To impose this gait, the rats were secured in a robotic interface that provided support against the direction of gravity and in the mediolateral direction, but behaved transparently in the forward direction. After completion of training, rats were chronically implanted with a micro-wire array spanning the left hindlimb motor cortex to record single and multi-unit activity, and bipolar electrodes into 10 muscles of the right hindlimb to monitor electromyographic signals. Whole-body kinematics, muscle activity, and neural signals were simultaneously recorded during execution of the trained tasks over multiple days of testing. Hindlimb kinematics, muscle activity, gait phases, and locomotor tasks were decoded using offline classification algorithms. Main results. We found that the stance and swing phases of gait and the locomotor tasks were detected with accuracies as robust as 90% in all rats. Decoded hindlimb kinematics and muscle activity exhibited a larger variability across rats and tasks. Significance. Our study shows that the rodent motor cortex contains useful information for lower limb neuroprosthetic development. However, brain-machine interfaces estimating gait phases or locomotor behaviors, instead of continuous variables such as limb joint positions or speeds, are likely to provide more robust control strategies for the design of such neuroprostheses.