Fast and accurate decoding of finger movements from ECoG through Riemannian features and modern machine learning techniques

Objective. Accurate decoding of individual finger movements is crucial for advanced prosthetic control. In this work, we introduce the use of Riemannian-space features and temporal dynamics of electrocorticography (ECoG) signal combined with modern machine learning (ML) tools to improve the motor decoding accuracy at the level of individual fingers. Approach. We selected a set of informative biomarkers that correlated with finger movements and evaluated the performance of state-of-the-art ML algorithms on the brain-computer interface (BCI) competition IV dataset (ECoG, three subjects) and a second ECoG dataset with a similar recording paradigm (Stanford, nine subjects). We further explored the temporal concatenation of features to effectively capture the history of ECoG signal, which led to a significant improvement over single-epoch decoding in both classification (p < 0.01) and regression tasks (p < 0.01). Main results. Using feature concatenation and gradient boosted trees (the top-performing model), we achieved a classification accuracy of 77.0% in detecting individual finger movements (six-class task, including rest state), improving over the state-of-the-art conditional random fields by 11.7% on the three BCI competition subjects. In continuous decoding of movement trajectory, our approach resulted in an average Pearson's correlation coefficient (r) of 0.537 across subjects and fingers, outperforming both the BCI competition winner and the state-of-the-art approach reported on the same dataset (CNN + LSTM). Furthermore, our proposed method features a low time complexity, with only

Learning to Detect Objects with Minimal Supervision

Karim Ali

Many classes of objects can now be successfully detected with statistical machine learning techniques. Faces, cars and pedestrians, have all been detected with low error rates by learning their appearance in a highly generic manner from extensive training sets. These recent advances have enabled the use of reliable object detection components in real systems, such as automatic face focusing functions on digital cameras. One key drawback of these methods, and the issue addressed here, is the prohibitive requirement that training sets contain thousands of manually annotated examples. We present three methods which make headway toward reducing labeling requirements and in turn, toward a tractable solution to the general detection problem. First, we propose a new learning strategy for object detection. The proposed scheme forgoes the need to train a collection of detectors dedicated to homogeneous families of poses, and instead learns a single classifier that has the inherent ability to deform based on the signal of interest. We train a detector with a standard AdaBoost procedure by using combinations of pose-indexed features and pose estimators. This allows the learning process to select and combine various estimates of the pose with features able to compensate for variations in pose without the need to label data for training or explore the pose space in testing. We validate our framework on three types of data: hand video sequences, aerial images of cars, as well as face images. We compare our method to a standard Boosting framework, with access to the same ground truth, and show a reduction in the false alarm rate of up to an order of magnitude. Where possible, we compare our method to the state-of-the art, which requires pose annotations of the training data, and demonstrate comparable performance. Second, we propose a new learning method which exploits temporal consistency to successfully learn a complex appearance model from a sparsely labeled training video. Our approach consists in iteratively improving an appearance-based model built with a Boosting procedure, and the reconstruction of trajectories corresponding to the motion of multiple targets. We demonstrate the efficiency of our procedure by learning a pedestrian detector from videos and a cell detector from microscopy image sequences. In both cases, our method is demonstrated to reduce the labeling requirement by one to two orders of magnitude. We show that in some instances, our method trained with sparse labels on a video sequence is able to outperform a standard learning procedure trained with the fully labeled sequence. Third, we propose a new active learning procedure which exploits the spatial structure of image data and queries entire scenes or frames of a video rather than individual examples. We extend the Query by Committee approach allowing it to characterize the most informative scenes that are to be selected for labeling. We show that an aggressive procedure which exhibits zero tolerance to target localization error performs as well as more sophisticated strategies taking into account the trade-off between missed detections and localization error. Finally, we combine this method with our two proposed approaches above and demonstrate that the resulting algorithm can properly perform car detection from a small set of annotated image as well as pedestrian detection from a handful of labeled video frames.

EPFL2012

Motor and cognitive brain circuits of brain-machine interactions

Silvia Marchesotti

Brain-machine interfaces (BMIs) allow the user to control an external device such as a robotic arm, a cursor, or an avatar in a virtual world through the real-time decoding of brain signals and without the involvement of the musculoskeletal system. Although BMIs hold great promise for providing motor-impaired patients with means of control and interaction with the external world, the neurocognitive mechanisms that are involved in the use of a BMI remain poorly understood. In addition, BMIs allow researchers to investigate and separate the brain processes underlying cognitive functions from those related to motor control and afferent sensory signals that reliably accompany movements. Thus, BMIs are powerful tools for basic and applied neuroscience research. The present work investigates different stages of a BMI-mediated action and targets the subjective, behavioral, and neural signals of BMI control based on electroencephalography (EEG) signals. More specifically, the main study of this thesis has focused on the development of a multimodal imaging platform that allows EEG-based BMI control inside the magnetic resonance imaging (MRI) scanner and during the acquisition of functional MRI (fMRI). This platform has enabled us to exploit both the high temporal resolution of EEG and the high spatial resolution of fMRI to precisely identify the brain mechanisms underlying BMI control and those reflecting the subjective feeling of being in control over a BMI-action (i.e., the sense of agency, SoA). In this study we found an extended cortico-subcortical network involved in operating a motor-imagery BMI. Overall BMI performance was associated with activity in a set of regions including contralateral premotor cortex and the posterior cingulate cortex. Finally, cortical midline regions and the basal ganglia were involved in the subjective sense of controlling the BMI. In a second study, we further investigated whether the ability to control a BMI relates to the ability to perform motor imagery. Despite decades of technical advances, effective BMI-control remains limited to a subset of users. We show that inter-subject variability in BMI proficiency is associated with differences in motor imagery accuracy as captured by subjective and behavioral measurements, pointing to a prominent role of kinesthetic rather than visual imagery. We also identified enhanced lateralized ÎŒ-band oscillations over sensorimotor cortices during motor imagery in high- versus low-aptitude BMI users. Finally, we developed a novel paradigm for joint BMI actions, allowing two users to be jointly engaged in BMI control; our preliminary data provide evidence in support of the hypothesis that joint actions yield BMI-performance improvement even in absence of a physical connection. Our findings also show that during joint BMI-control the SoA over the imagery-mediated actions is significantly enhanced, and is affected by BMI abilities. This work show the potential of applying a multimodal imaging approach to the field of BMI: exploiting our EEG-BMI-fMRI platform we were able to identify, the neural mechanisms involved in motor and cognitive aspects of imagery-mediated BMI. Furthermore, our results enable us to better understand the subjective and behavioral aspects of BMI-actions, and reveal strategies that could potentially reinforce BMI control. Our findings can ultimately be of relevance for the field of neuroprosthetics and make BMI more accessible to a broader range of users.

EPFL2016

FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

Effrosyni Simou

In several machine learning tasks for graph structured data, the graphs under consideration may be composed of a varying number of nodes. Therefore, it is necessary to design pooling methods that aggregate the graph representations of varying size to representations of fixed size which can be used in downstream tasks, such as graph classification. Existing graph pooling methods offer no guarantee with regards to the similarity of a graph representation and its pooled version. In this work we address this limitation by proposing FlowPool, a pooling method that optimally preserves the statistics of a graph representation to its pooled counterpart by minimizing their Wasserstein distance. This is achieved by performing a Wasserstein gradient flow with respect to the pooled graph representation. We propose a versatile implementation of our method which can take into account the geometry of the representation space through any ground cost. This implementation relies on the computation of the gradient of the Wasserstein distance with recently proposed implicit differentiation schemes. Our pooling method is amenable to automatic differentiation and can be integrated in end-to-end deep learning architectures. Further, FlowPool is invariant to permutations and can therefore be combined with permutation equivariant feature extraction layers in GNNs in order to obtain predictions that are independent of the ordering of the nodes. Experimental results demonstrate that our method leads to an increase in performance compared to existing pooling methods when evaluated in graph classification tasks.