Decoding bipedal locomotion from the rat sensorimotor cortex

Objective. Decoding forelimb movements from the firing activity of cortical neurons has been interfaced with robotic and prosthetic systems to replace lost upper limb functions in humans. Despite the potential of this approach to improve locomotion and facilitate gait rehabilitation, decoding lower limb movement from the motor cortex has received comparatively little attention. Here, we performed experiments to identify the type and amount of information that can be decoded from neuronal ensemble activity in the hindlimb area of the rat motor cortex during bipedal locomotor tasks. Approach. Rats were trained to stand, step on a treadmill, walk overground and climb staircases in a bipedal posture. To impose this gait, the rats were secured in a robotic interface that provided support against the direction of gravity and in the mediolateral direction, but behaved transparently in the forward direction. After completion of training, rats were chronically implanted with a micro-wire array spanning the left hindlimb motor cortex to record single and multi-unit activity, and bipolar electrodes into 10 muscles of the right hindlimb to monitor electromyographic signals. Whole-body kinematics, muscle activity, and neural signals were simultaneously recorded during execution of the trained tasks over multiple days of testing. Hindlimb kinematics, muscle activity, gait phases, and locomotor tasks were decoded using offline classification algorithms. Main results. We found that the stance and swing phases of gait and the locomotor tasks were detected with accuracies as robust as 90% in all rats. Decoded hindlimb kinematics and muscle activity exhibited a larger variability across rats and tasks. Significance. Our study shows that the rodent motor cortex contains useful information for lower limb neuroprosthetic development. However, brain-machine interfaces estimating gait phases or locomotor behaviors, instead of continuous variables such as limb joint positions or speeds, are likely to provide more robust control strategies for the design of such neuroprostheses.

Rich and Robust Bio-Inspired Locomotion Control for Humanoid Robots

Nicolas Benoît Dominique Van der Noot

Bipedal locomotion is a challenging task in the sense that it requires to maintain dynamic balance while steering the gait in potentially complex environments. Yet, humans usually manage to move without any apparent difficulty, even on rough terrains. This requires a complex control scheme which is far from being understood. In this thesis, we take inspiration from the impressive human walking capabilities to design neuromuscular controllers for humanoid robots. More precisely, we control the robot motors to reproduce the action of virtual muscles commanded by stimulations (i.e. neural signals), similarly to what is done during human locomotion. Because the human neural circuitry commanding these muscles is not completely known, we make hypotheses about this control scheme to simplify it and progressively refine the corresponding rules. This thesis thus aims at developing new walking algorithms for humanoid robots in order to obtain fast, human-like and energetically efficient gaits. In particular, gait robustness and richness are two key aspects of this work. In other words, the gaits developed in the thesis can be steered by an external operator, while being resistant to external perturbations. This is mainly tested during blind walking experiments on COMAN, a 95 cm tall humanoid robot. Yet, the proposed controllers can be adapted to other humanoid robots. In the beginning of this thesis, we adapt and port an existing reflex-based neuromuscular model to the real COMAN platform. When tested in a 2D simulation environment, this model was capable of reproducing stable human-like locomotion. By porting it to real hardware, we show that these neuromuscular controllers are viable solutions to develop new controllers for robotics locomotion. Starting from this reflex-based model, we progressively iterate and transform the stimulation rules to add new features. In particular, gait modulation is obtained with the inclusion of a central pattern generator (CPG), a neural circuit capable of producing rhythmic patterns of neural activity without receiving rhythmic inputs. Using this CPG, the 2D walker controllers are incremented to generate gaits across a range of forward speeds close to the normal human one. By using a similar control method, we also obtain 2D running gaits whose speed can be controlled by a human operator. The walking controllers are later extended to 3D scenarios (i.e. no motion constraint) with the capability to adapt both the forward speed and the heading direction (including steering curvature). In parallel, we also develop a method to automatically learn stimulation networks for a given task and we study how flexible feet affect the gait in terms of robustness and energy efficiency. In sum, we develop neuromuscular controllers generating human-like gaits with steering capabilities. These controllers recruit three main components: (i) virtual muscles generating torque references at the joint level, (ii) neural signals commanding these muscles with reflexes and CPG signals, and (iii) higher level commands controlling speed and heading. Interestingly, these developments target humanoid robots locomotion but can also be used to better understand human locomotion. In particular, the recruitment of a CPG during human locomotion is still a matter open to debate. This question can thus benefit from the experiments performed in this thesis.

EPFL2017

Integration of a virtual reality based arm in primary somatosensory cortex

Hannes Bleuler, Miguel Nicolelis, Solaiman Shokur

Abstract: Recent advances in brain-machine interfaces (BMIs) have demonstrated the possibility of motor neuroprosthetics directly controlled by brain activity. Ideally neuroprosthetic limbs should be integrated in the body schema of the subject. To explore the ways to enhance such incorporation, we recorded modulations of neuronal ensemble activity in the primary somatosensory (S1) cortex during tactile stimulation simulated in virtual reality (VR) under conditions known to evoke a rubber-hand illusion. A realistic 3D mesh represented monkey body in VR. The monkey’s arms were hidden by an opaque plate and virtual arms projected on the plate. A robotic brush, also hidden from the monkey, touched various locations on forearms of the monkey and was synchronized with a virtual brush touching the projected VR arms. Additionally, we implemented tactile stimulation with air puffs. We have tested various combination of tactile (physical touch), visual (VR arm being touched) and sound (robotic brush touching the arm) inputs: synchronous tactile and visual (T-VR), tactile without visual (T), and visual only (VR). Neuronal ensemble activity was recorded from S1 and primary motor cortex (M1). We found differences in both S1 and M1 activities across the stimulation types. In particular S1 responses to T-VR were stronger than for T. Moreover, S1 neurons were modulated during visual stimulation without touch (VR), suggesting S1 activation as a neuronal mechanism of the rubber-hand illusion. Further, we decoded stimulation parameters from the activity of large neuronal populations. These results suggest a flexible and distributed representation of somatosensory information in the cortex, which can be modified by visual feedback from the body and/or artificial actuators.

2010

Active tactile exploration using a brain-machine-brain interface

Hannes Bleuler, Miguel Nicolelis, Solaiman Shokur

Brain–machine interfaces1, 2 use neuronal activity recorded from the brain to establish direct communication with external actuators, such as prosthetic arms. It is hoped that brain–machine interfaces can be used to restore the normal sensorimotor functions of the limbs, but so far they have lacked tactile sensation. Here we report the operation of a brain–machine–brain interface (BMBI) that both controls the exploratory reaching movements of an actuator and allows signalling of artificial tactile feedback through intracortical microstimulation (ICMS) of the primary somatosensory cortex. Monkeys performed an active exploration task in which an actuator (a computer cursor or a virtual-reality arm) was moved using a BMBI that derived motor commands from neuronal ensemble activity recorded in the primary motor cortex. ICMS feedback occurred whenever the actuator touched virtual objects. Temporal patterns of ICMS encoded the artificial tactile properties of each object. Neuronal recordings and ICMS epochs were temporally multiplexed to avoid interference. Two monkeys operated this BMBI to search for and distinguish one of three visually identical objects, using the virtual-reality arm to identify the unique artificial texture associated with each. These results suggest that clinical motor neuroprostheses might benefit from the addition of ICMS feedback to generate artificial somatic perceptions associated with mechanical, robotic or even virtual prostheses.

2011

Rich and Robust Bio-Inspired Locomotion Control for Humanoid Robots

Nicolas Benoît Dominique Van der Noot

EPFL2017