Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques

Background: One of the current challenges in brain-machine interfacing is to characterize and decode upper limb kinematics from brain signals, e.g. to control a prosthetic device. Recent research work states that it is possible to do so based on low frequency EEG components. However, the validity of these results is still a matter of discussion. In this paper, we assess the feasibility of decoding upper limb kinematics from EEG signals in center-out reaching tasks during passive and active movements. Methods: The decoding of arm movement was performed using a multidimensional linear regression. Passive movements were analyzed using the same methodology to study the influence of proprioceptive sensory feedback in the decoding. Finally, we evaluated the possible advantages of classifying reaching targets, instead of continuous trajectories. Results: The results showed that arm movement decoding was significantly above chance levels. The results also indicated that EEG slow cortical potentials carry significant information to decode active center-out movements. The classification of reached targets allowed obtaining the same conclusions with a very high accuracy. Additionally, the low decoding performance obtained from passive movements suggests that discriminant modulations of low-frequency neural activity are mainly related to the execution of movement while proprioceptive feedback is not sufficient to decode upper limb kinematics. Conclusions: This paper contributes to the assessment of feasibility of using linear regression methods to decode upper limb kinematics from EEG signals. From our findings, it can be concluded that low frequency bands concentrate most of the information extracted from upper limb kinematics decoding and that decoding performance of active movements is above chance levels and mainly related to the activation of cortical motor areas. We also show that the classification of reached targets from decoding approaches may be a more suitable real-time methodology than a direct decoding of hand position.

Brain-actuated gait trainer with visual and proprioceptive feedback

Mohamed Bouri, Ricardo Andres Chavarriaga Lozano, Kyuhwa Lee, Dong Liu

Objective. Brain-machine interfaces (BMIs) have been proposed in closed-loop applications for neuromodulation and neurorehabilitation. This study describes the impact of different feedback modalities on the performance of an EEG-based BMI that decodes motor imagery (MI) of leg flexion and extension. Approach. We executed experiments in a lower-limb gait trainer (the legoPress) where nine able-bodied subjects participated in three consecutive sessions based on a crossover design. A random forest classifier was trained from the offline session and tested online with visual and proprioceptive feedback, respectively. Post-hoc classification was conducted to assess the impact of feedback modalities and learning effect (an improvement over time) on the simulated trial-based performance. Finally, we performed feature analysis to investigate the discriminant power and brain pattern modulations across the subjects. Main Results. (i) For real-time classification, the average accuracy was 62.33 ± 4.95% and 63.89 ± 6.41% for the two online sessions. The results were significantly higher than chance level, demonstrating the feasibility to distinguish between MI of leg extension and flexion. (ii) For post-hoc classification, the performance with proprioceptive feedback (69.45 ± 9.95%) was significantly better than with visual feedback (62.89 ± 9.20%), while there was no significant learning effect. (iii) We reported individual discriminate features and brain patterns associated to each feedback modality, which exhibited differences between the two modalities although no general conclusion can be drawn. Significance. The study reported a closed-loop brain-controlled gait trainer, as a proof of concept for neurorehabilitation devices. We reported the feasibility of decoding lower-limb movement in an intuitive and natural way. As far as we know, this is the first online study discussing the role of feedback modalities in lower-limb MI decoding. Our results suggest that proprioceptive feedback has an advantage over visual feedback, which could be used to improve robot-assisted strategies for motor training and functional recovery.

Institute of Physics2017

Graph-based image representation learning

Renata Khasanova

Though deep learning (DL) algorithms are very powerful for image processing tasks, they generally require a lot of data to reach their full potential. Furthermore, there is no straightforward way to impose various properties, given by the prior knowledge about the target task, on the features extracted by a DL model. Therefore, in this thesis we propose several techniques that rely on the power of graph representations to embed prior knowledge inside the learning process. This allows to reduce the solution space and leads to faster optimization convergence and higher accuracy in the representation learning. In our first work, inspired by the ability of a human to correctly classify rotated, shifted or flipped objects, we propose an algorithm that permits to inherently encode invariance to isometric transformations of objects in an image. Our DL architecture is based on graph representations and consists of three novel layers, which we refer to as graph convolutional, dynamic pooling and statistical layers. Our experiments on the image classification tasks show that our network correctly recognizes isometrically transformed objects even though such types of transformation are not seen by the network at training time. Standard DL techniques are typically not able to succeed in solving such a problem without extensive data augmentation. Then, we propose to exploit the properties of graph-based approaches to efficiently process images with various types of projective geometry. In particular, we are interested in increasingly popular omnidirectional cameras, which have a 360 degree field of view. Despite their effectiveness, such cameras create images with specific geometric properties, which require special techniques for efficient processing. We propose an efficient way of adjusting the weights of the graph edges to adapt the filter responses to the geometric image properties introduced by omnidirectional cameras. Our experiments prove that using the proposed graph with properly adjusted edge weights permits to reach better performance as compared to using regular grid graph with equal weights. Finally, the approach described above relies on the isotropic filters, which work well within our transformation invariant architecture for image classification. However, for other problems (e.g. image compression) or even when used without dynamic pooling and statistical layers that are defined within the proposed architecture, these filters are unable to efficiently encode the information about the object. Thus, we introduce a different technique based on anisotropic filters that adapt their shape and size according to the omnidirectional image geometry. The main advantage of this approach compared to the previous one is the ability to encode the orientation of an image pattern, which is important for various tasks such as image compression. Our experiments show that our approach adapts to different image projective geometries and achieves state-of-the-art performance on image classification and compression tasks. Overall we propose several methods, which combine the power of DL and graph signal processing towards incorporating prior information about the target task inside the optimization procedure. We hope that the research efforts presented in this thesis will help the development of efficient DL algorithms that can use various types of prior knowledge to make them efficient even when the available training data is scarce.

EPFL2019

Uncovering the neural encoding of behavior in the adult Drosophila motor system

Chin-Lin Chen

One of the most important goals in neuroscience research has always been to understand how animals control their behavior. However, the long focus on the role of brain neurons in behavioral control might be missing the full story. In fact, brain-wide fluctuations in neural activity are highly correlated to the behavioral state, suggesting a closed-loop system where the behavioral command and the body state interact to interpret sensory inputs and dynamically guide action selection. However, to date, little is known about how and where information about the behavioral state is generated. In principle, information about the behavioral state is a relatively high-level encoding of the body state derived from the integration of signals from primary sensors and the local network. Local motor circuits residing in the spinal cord of mammals or the ventral nerve cord (VNC) of insects may give rise to the behavioral state and pass to the brain via brain-projecting interneurons---termed ascending neurons (ANs). However, what information ANs carry and where this information originates is still a mystery due to the technical challenges in measuring these signals in behaving animals. Therefore, it is critical to develop tools to track neuron identity and record neuronal activity during behavior to crack the functional network. In this work, a method is introduced for imaging the motor circuit in the VNC of Drosophila melanogaster. This preparation demonstrates how to retain fly limb function after dissection and allows simultaneous recording of a tethered fly's behavior and its neural activity in the VNC. Incorporating this methodology with other approaches can verify the conclusions from the experiments such as those manipulating neuronal excitability, and motivate further investigation into neuronal function in specific types of behavior. Since it has long been theorized that ANs are the cellular basis of the behavioral state information sent in the brain, the technique introduced in this work was designed to resolve questions relating to what type of information ANs convey. Here, a large-scale functional screen was performed to relate neural activity, behavior, and neuronal morphology for 247 genetically identified ANs in the adult fly. This is the first large-scale screen of neural activity in a behaving animal, and this work uncovered three fundamental features of AN populations. First, rather than low-level limb movements, ANs encode high-level behaviors like walking, grooming, and resting. Second, ANs project nearly exclusively to an integrative sensory center (AVLP) and an action selection hub (GNG). Remarkably, the encodings projected to these distinct areas reflect their potential roles in contextualizing sensory cues (AVLP) and guiding future actions (GNG). Third, the behavioral encodings of ANs are closely linked to their motor system patterning. In conclusion, this work provides an indispensable tool to study the motor system beyond the brain network by overcoming a major technical barrier in in-vivo recording of the VNC of a behaving fly. This tool was used to help resolve what is encoded by ANs as a higher-level behavioral state, suggesting that the primary cellular origin of the behavioral information that is used to modulate brain function comes from the VNC. Overall, the technique and the findings on AN encoding should inspire follow-up studies to uncover additional neural mechanisms used in adaptive behavior generation.

EPFL2022