Real-Time Cognitive Workload Monitoring Based on Machine Learning Using Physiological Signals in Rescue Missions

High levels of cognitive workload decreases human's performance and leads to failures with catastrophic outcomes in risky missions. Today, reliable cognitive workload detection presents a common major challenge, since the workload is not directly observable. However, cognitive workload affects several physiological signals that can be measured non-invasively. The main goal of this work is to develop a reliable machine learning algorithm to identify the cognitive workload induced during rescue missions, which is evaluated through drone control simulation experiments. In addition, we aim to minimize the computing resources usage while maximizing the cognitive workload detection accuracy for a reliable real-time operation. We perform an experiment in which 24 subjects played a rescue mission simulator while respiration, electrocardiogram, photoplethysmogram, and skin temperature signals were measured. State-of-the-art feature-based machine learning algorithms are investigated for cognitive workload characterization using learning curves, data augmentation, and cross-validation techniques. The best classification algorithm is selected, optimized, and the most informative features are selected. Finally, the generalization power of the optimized model is evaluated on an unseen test set. We obtain an accuracy level of 86% on the new unseen datasets using the proposed and optimized eXtreme Gradient Boosting (XGB) algorithm. Then, we reduce the complexity of the machine learning model for future implementation on resource-constrained wearable embedded systems, by optimizing the model and selecting the 26 most important features. Overall, a generalizable and low-complexity machine learning model for cognitive workload detection based on physiological signals is presented for the first time in the literature.

Reaching over to grasp an item is arguably the most commonly used motor skill by humans. Even under sudden perturbations, humans seem to react rapidly and adapt their motion to guarantee success. Despite the apparent ease and frequency with which we use this ability, a complete understanding of the underlying mechanisms cannot be claimed. It is partly due to such incomplete knowledge that adaptive robot motion for reaching and grasping under perturbations is not perfectly achieved. In this thesis, we take the discriminative approach for modelling trajectories of reach-to-grasp motion from expert demonstrations. Throughout this thesis, we will employ time-independent (autonomous) flow based representations to learn reactive motion controllers which can then be ported onto robots. This thesis is divided into three main parts. The first part is dedicated to biologically inspired modelling of reach-to-grasp motions with respect to the hand-arm coupling. We build upon previous work in motion modelling using autonomous dynamical systems (DS) and present a coupled dynamical system (CDS) model of these two subsystems. The coupled model ensures satisfaction of the constraints between the hand and the arm subsystems which are critical to the success of a reach-to-grasp task. Moreover, it reduces the complexity of the overall motion planning problem as compared to considering a combined problem for the hand and the arm motion. In the second part we extend the CDS approach to incorporate multiple grasping points. Such a model is beneficial due to the fact that many daily life objects afford multiple grasping locations on their surface. We combine a DS based approach with energy-function learning to learn a multiple attractor dynamical system where the attractors are mapped to the desired grasping points. We present the Augmented-SVM (ASVM) model that combines the classical SVM formulation with gradient constraints arising from the energy function to learn the desired dynamical function for motion generation. In the last part of this thesis, we address the problem of inverse-kinematics and obstacle avoidance by combining our flow-based motion generator with global configuration-space planners. We claim that the two techniques complement each other. On one hand, the fast reactive nature of our flow based motion generator can used to guide the search of a randomly exploring random tree (RRT) based global planner. On the other hand, global planners can efficiently handle arbitrary obstacles and avoid local minima present in the dynamical function learned from demonstrations. We show that combining the information from demonstrations with global planning in the form of a energy-map considerably decreases the computational complexity of state-of-the-art sampling based planners. We believe that this thesis has the following contributions to Robotics and Machine Learning. First, we have developed algorithms for fast and adaptive motion generation for reach-grasp motions. Second, we formulated an extension to the classical SVM formulation that takes into account the gradient information from data. We showed that instead of being limited as a classifier or a regressor, the SVM framework can be used as a more general function approximation technique. Lastly, we have combined our local methods with global approaches for planning to achieve arbitrary obstacle avoidance and considerable reduction in the computation complexity of the global planners.

EPFL2014

OrthoNet: Multilayer Network Data Clustering

Giovanni Chierchia, Mireille El Gheche, Pascal Frossard

Network data appears in very diverse applications, like biological, social, or sensor networks. Clustering of network nodes into categories or communities has thus become a very common task in machine learning and data mining. Network data comes with some information about the network edges. In some cases, this network information can even be given with multiple views or layers, each one representing a different type of relationship between the network nodes. Increasingly often, network nodes also carry a feature vector. We propose in this paper to extend the node clustering problem, that commonly considers only the network information, to a problem where both the network information and the node features are considered together for learning a clustering-friendly representation of the feature space. Specifically, we design a generic two-step algorithm for multilayer network data clustering. The first step aggregates the different layers of network information into a graph representation given by the geometric mean of the network Laplacian matrices. The second step uses a neural net to learn a feature embedding that is consistent with the structure given by the network layers. We propose a novel algorithm for efficiently training the neural net via gradient descent, which encourages the neural net outputs to span the leading eigenvectors of the aggregated Laplacian matrix, in order to capture the pairwise interactions on the network, and provide a clustering-friendly representation of the feature space. We demonstrate with an extensive set of experiments on synthetic and real datasets that our method leads to a significant improvement w.r.t. state-of-the-art multilayer graph clustering algorithms, as it judiciously combines nodes features and network information in the node embedding algorithms.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Learning Reach-to-Grasp Motions From Human Demonstrations

Ashwini Shukla

EPFL2014

Real-Time Cognitive Workload Monitoring Based on Machine Learning Using Physiological Signals in Rescue Missions

Adriana Arza Valdes, David Atienza Alonso, Niloofar Momeni