Motor state transitions and Breathing in Brain Machine Interfaces

Brain-Machine interfaces aim to create a direct neural link between user's brain and machines. This goal has pushed scientists to investigate a large spectrum of applications in the realm of assistive and rehabilitation technologies. However, despite great progress, the possibility of being in full control of a device with our brain is still far to be reached. One of the primary limitations of BMIs is the notion of self-paced control, which states that a user should be able to activate or deactivate such interface by mere modulation of brain activity. Many studies have investigated this question, focusing notably on the use of endogenous signals (such as Event-Related Desynchronization/Synchronization, ERD/S) to control the activation of these interfaces in natural interaction scenarios. This question is, however, largely ignored when looking at their interruption. Hence, in this first part of the thesis, my work was devoted to investigate how a BMI user would be able to control the interruption of non-invasive BMIs based on Motor Imagery (MI), a paradigm promoting natural and endogenous signals that can be used for BMI control. I investigated the decoding of motor termination as well as its correlates characterized by the post-movement ERS phenomenon. Specifically, this part of the thesis aimed to study (i) the feasibility to decode motor termination from a specific neural correlate as well as the adaptation from the user in closed-loop scenarios, (ii) the benefits of using motor termination correlates to detect the stopping process, and (iii) the effect of BMI effectors such as an exoskeleton on the detection of motor termination correlates. The obtained results provide new insights into closed-loop decoding of motor termination. In particular, I show that BMI users exhibit an adaptation of their EEG correlates enabling them to have a reliable control when switching off a BMI in closed-loop scenarios. Second, I show that the decoding of motor termination is a particular process different from a resting state and, hence, should be decoded independently so as to achieve a faster and more robust detection. Finally, I investigate the use of BMI effectors with respect to the decoding of motor termination showing an effect of the effectors on the correlates of motor termination. However, due to the nature of these correlates, motor termination can still be reliably decoded with a similar latency. In the second part of my thesis, I also investigate how the interoceptive system affects BMI and particularly how breathing signals affect BMI based on MI. Breathing has been shown to have a key effect on numerous human functions, including, perceptual, cognitive and motor functions. However, currently not much is known about the role of respiratory signals in BMI and such signals have rather been considered as physiological noise. In my thesis, I evaluated how the breathing process affects the correlates of MI (ERD in mu and beta bands) as well as actual BMI performance. The results provide an extensive analysis regarding the effect of the breathing cycle specifically on mu-ERDs, showing stronger ERD during the late expiration phase. Moreover, I identified a link between breathing and BMI performance and propose that breathing signals are a valuable predictor for BMI performance highlighting the importance of monitoring such signals and, more generally, present the interoceptive system as a key component of motor preparation and motor imagery

Uncovering the neural encoding of behavior in the adult Drosophila motor system

Chin-Lin Chen

One of the most important goals in neuroscience research has always been to understand how animals control their behavior. However, the long focus on the role of brain neurons in behavioral control might be missing the full story. In fact, brain-wide fluctuations in neural activity are highly correlated to the behavioral state, suggesting a closed-loop system where the behavioral command and the body state interact to interpret sensory inputs and dynamically guide action selection. However, to date, little is known about how and where information about the behavioral state is generated. In principle, information about the behavioral state is a relatively high-level encoding of the body state derived from the integration of signals from primary sensors and the local network. Local motor circuits residing in the spinal cord of mammals or the ventral nerve cord (VNC) of insects may give rise to the behavioral state and pass to the brain via brain-projecting interneurons---termed ascending neurons (ANs). However, what information ANs carry and where this information originates is still a mystery due to the technical challenges in measuring these signals in behaving animals. Therefore, it is critical to develop tools to track neuron identity and record neuronal activity during behavior to crack the functional network. In this work, a method is introduced for imaging the motor circuit in the VNC of Drosophila melanogaster. This preparation demonstrates how to retain fly limb function after dissection and allows simultaneous recording of a tethered fly's behavior and its neural activity in the VNC. Incorporating this methodology with other approaches can verify the conclusions from the experiments such as those manipulating neuronal excitability, and motivate further investigation into neuronal function in specific types of behavior. Since it has long been theorized that ANs are the cellular basis of the behavioral state information sent in the brain, the technique introduced in this work was designed to resolve questions relating to what type of information ANs convey. Here, a large-scale functional screen was performed to relate neural activity, behavior, and neuronal morphology for 247 genetically identified ANs in the adult fly. This is the first large-scale screen of neural activity in a behaving animal, and this work uncovered three fundamental features of AN populations. First, rather than low-level limb movements, ANs encode high-level behaviors like walking, grooming, and resting. Second, ANs project nearly exclusively to an integrative sensory center (AVLP) and an action selection hub (GNG). Remarkably, the encodings projected to these distinct areas reflect their potential roles in contextualizing sensory cues (AVLP) and guiding future actions (GNG). Third, the behavioral encodings of ANs are closely linked to their motor system patterning. In conclusion, this work provides an indispensable tool to study the motor system beyond the brain network by overcoming a major technical barrier in in-vivo recording of the VNC of a behaving fly. This tool was used to help resolve what is encoded by ANs as a higher-level behavioral state, suggesting that the primary cellular origin of the behavioral information that is used to modulate brain function comes from the VNC. Overall, the technique and the findings on AN encoding should inspire follow-up studies to uncover additional neural mechanisms used in adaptive behavior generation.

EPFL2022

Motor and cognitive brain circuits of brain-machine interactions

Silvia Marchesotti

Brain-machine interfaces (BMIs) allow the user to control an external device such as a robotic arm, a cursor, or an avatar in a virtual world through the real-time decoding of brain signals and without the involvement of the musculoskeletal system. Although BMIs hold great promise for providing motor-impaired patients with means of control and interaction with the external world, the neurocognitive mechanisms that are involved in the use of a BMI remain poorly understood. In addition, BMIs allow researchers to investigate and separate the brain processes underlying cognitive functions from those related to motor control and afferent sensory signals that reliably accompany movements. Thus, BMIs are powerful tools for basic and applied neuroscience research. The present work investigates different stages of a BMI-mediated action and targets the subjective, behavioral, and neural signals of BMI control based on electroencephalography (EEG) signals. More specifically, the main study of this thesis has focused on the development of a multimodal imaging platform that allows EEG-based BMI control inside the magnetic resonance imaging (MRI) scanner and during the acquisition of functional MRI (fMRI). This platform has enabled us to exploit both the high temporal resolution of EEG and the high spatial resolution of fMRI to precisely identify the brain mechanisms underlying BMI control and those reflecting the subjective feeling of being in control over a BMI-action (i.e., the sense of agency, SoA). In this study we found an extended cortico-subcortical network involved in operating a motor-imagery BMI. Overall BMI performance was associated with activity in a set of regions including contralateral premotor cortex and the posterior cingulate cortex. Finally, cortical midline regions and the basal ganglia were involved in the subjective sense of controlling the BMI. In a second study, we further investigated whether the ability to control a BMI relates to the ability to perform motor imagery. Despite decades of technical advances, effective BMI-control remains limited to a subset of users. We show that inter-subject variability in BMI proficiency is associated with differences in motor imagery accuracy as captured by subjective and behavioral measurements, pointing to a prominent role of kinesthetic rather than visual imagery. We also identified enhanced lateralized ÎŒ-band oscillations over sensorimotor cortices during motor imagery in high- versus low-aptitude BMI users. Finally, we developed a novel paradigm for joint BMI actions, allowing two users to be jointly engaged in BMI control; our preliminary data provide evidence in support of the hypothesis that joint actions yield BMI-performance improvement even in absence of a physical connection. Our findings also show that during joint BMI-control the SoA over the imagery-mediated actions is significantly enhanced, and is affected by BMI abilities. This work show the potential of applying a multimodal imaging approach to the field of BMI: exploiting our EEG-BMI-fMRI platform we were able to identify, the neural mechanisms involved in motor and cognitive aspects of imagery-mediated BMI. Furthermore, our results enable us to better understand the subjective and behavioral aspects of BMI-actions, and reveal strategies that could potentially reinforce BMI control. Our findings can ultimately be of relevance for the field of neuroprosthetics and make BMI more accessible to a broader range of users.

EPFL2016

Neuroprosthetics to support impaired hand functions: A multidisciplinary approach combining brain-machine interfaces and wearable exoskeletons

Luca Randazzo

Neuroprosthetics, the discipline that aims at interfacing neural systems to artificially engineered devices, has witnessed in recent years important advancements towards the ultimate goal of augmenting and restoring human functions through technology and artificial systems. Research in this field covers a wide range of efforts, from advancing our basic understanding of the neural processes involved into human sensory, motor and cognitive functions, to developing artificial intelligence algorithms for their decoding, to the physical interfacing between artificial devices and the human nervous system and biomechanics. Ultimately, this discipline aims at fusing these systems in order to restore, replace and rehabilitate lost and impaired functions due to motor disabling conditions or traumatic accidents. In this thesis, I present a multidisciplinary approach that aimed at developing a neuroprosthesis for the assistance and restoration of hand functions impaired by neurological disorders or traumatic accidents, such as cerebrovascular accidents and spinal cord injuries. These efforts encompassed the conceptualization and development of a robotic hand exoskeleton and brain-machine interface (BMI) algorithms for its control. Specifically, this thesis focused on the design, development and testing of (i) a novel mechatronic hand exoskeleton to assist hand motions within domestic and clinical settings, (ii) non-invasive BMI approaches based on electroencephalography (EEG) to decode neural correlates of intended hand movements, and on (iii) the closed-loop integration of the proposed exoskeleton and BMI for the sake of providing continuous feedback about ongoing neural modulations through hand motions and to trigger sensorimotor rehabilitation within clinical scenarios. Results showed that the proposed mechatronic system can successfully control hand opening and closing within a fully wearable, portable and lightweight package. The system was tested with users who suffered from motor disabling impairments, showing that it could help them in performing several activities of daily living for the first time since their accidents. From a brain-machine interfacing perspective, this work shows how imagined hand movements can be decoded, through EEG, in parallel with exoskeleton-induced motions, with important implications for the development of more embodied human-machine interaction protocols. Finally, this work shows how the closed-loop control of exoskeleton motions by means of the decoded ongoing neural activity enhances the discriminability of sensorimotor neural patterns and improves the performance of the brain-machine control channel. Overall, the results presented here represent important advancements within the field of neuroprosthetics, with interesting implications for the development of assistive exoskeletal technologies and non-invasive brain-machine interfaces aimed at controlling such systems in clinical or domestic settings, for both assistive and neurorehabilitative purposes.

EPFL2018