MBioTracker: Multimodal Self-Aware Bio-Monitoring Wearable System for Online Workload Detection

Cognitive workload affects operators' performance principally in high-risk or time-demanding situations and when multitasking is required. An online cognitive workload monitoring system can provide valuable inputs to decision-making instances, such as the operator's state of mind and resulting performance. Therefore, it can allow potential adaptive support to the operator. This work presents a new design of a wearable embedded system for online cognitive workload monitoring. This new wearable system consists of, on the hardware side, a multi-channel physiological signals acquisition (respiration cycles, heart rate, skin temperature, and pulse waveform) and a low-power processing platform. Further, on the software side, our wearable embedded system includes a novel energy-aware bio-signal processing algorithm. We also use the concept of application self-awareness to enable energy-scalable embedded machine learning algorithms and methods for online subjects' cognitive workload monitoring. Our results show that this new wearable system can continuously monitor multiple bio-signals, compute their key features, and provide reliable detection of high and low cognitive workload levels with a time resolution of 1 minute and a battery lifetime of 14.58h in our experimental conditions. It achieves a detection accuracy of 76.6% (2.6% lower than analogous offline computer-based analysis) with a sensitivity of 77.04% and a specificity of 81.75%, on a simulated drone rescue mission task. Moreover, by applying our self-aware monitoring to exploit different energy-scalable modes, we can increase battery lifetime by 51.6% (up to 22.11 hours) while incurring an insignificant accuracy loss of 1.07%.

Wearable Technologies for Embodied Human-Robot Interaction

Carine Rognon

Robotic teleoperation is fundamental to augment the resilience, precision, and force of robots with the cognition of the operator. However, current interfaces, such as joysticks and remote controllers, are often complicated to handle since they require cognitive effort and learned skills. Wearable interfaces can enable more natural and intuitive interactions with robots, which would make robotic teleoperation accessible to a larger population of users for demanding tasks, such as manipulation or search-and-rescue. The aim of this thesis is to explore solutions to simplify our interactions with robots promoting their access to a broader range of the population. To achieve this, we are presenting a soft upper body exoskeleton, called the FlyJacket, for the bidirectional control of drones. Drones can greatly benefit us as they extend our perception and range of action. The exoskeleton controls a drone by recording torso movement and, through embedded haptic feedback devices, renders either kinesthetic guidance to improve the flight performance or tactile feedback to render the sensation of flying. We developed and tested an interface to control both a simulated and a real drone. The FlyJacket is a soft exoskeleton with arm support conceived to address the challenges of adapting to different morphologies and supporting the user during flight to prevent fatigue. We demonstrated that this novel interface allowed more consistent performance than when performing the same task with a remote controller and users felt more immersed into the flight. Interacting with a robot can be greatly enhanced by having multiple channels of sensorial feedback to increase the awareness of the operator. Information on the state of the drone can be intuitively rendered with haptic feedback. To create this bidirectional interaction with the drone, the two types of haptic feedback - kinesthetic and tactile - have been explored. Kinesthetic feedback was implemented with a cable-driven system to give guidance to the userâs torso position. Performing user studies, we could determine that the embedded guidance improved the flight performance and that a quadratically shaped force feedback curve was the most adequate profile to guide the user. We also established the minimal force difference, defined the perceived magnitude of this system and studied the learning process of users. Tactile feedback was investigated to render the sensation of flying by enhancing flight awareness, realism and immersion. To this end, we developed and embedded a new type of soft actuator that was compliant and lightweight such that it remained wearable and portable. Four devices, placed on the torso, provided feedback by compressing closed air pouches against the skin rendering the sensation of air pressure. A mechanical model and simulation of the pouch device were developed to determine appropriate parameters. We evaluated whether it conveyed useful information to the user and whether it enhanced the experience of flying. We demonstrated that users were able to understand the direction of the cues without prompting, could distinguish the cues quickly, and do so with high accuracy. The device was also used in a simulated flight task and users indicated that it increased the flight realism. We believe that the contributions of this thesis provides insights to the design of intuitive interfaces for human-robot interaction and increases their accessibility to a wider range of the population.

EPFL2019

CAFS: Cost-Aware Features Selection Method for Multimodal Stress Monitoring on Wearable Devices

Adriana Arza Valdes, David Atienza Alonso, João Pedro de Matos Rodrigues, Niloofar Momeni, Maria del Carmen Sandi Perez

Objective: Today, stress monitoring on wearable devices is challenged by the tension between high-detection accuracy and battery lifetime driven by multimodal data acquisition and processing. Limited research has addressed the classification cost on multimodal wearable sensors, particularly when the features are cost-dependent. Thus, we design a Cost-Aware Feature Selection (CAFS) methodology that trades-off between prediction-power and energy-cost for multimodal stress monitoring. Methods: CAFS selects the most important features under different energy-constraints, which allows us to obtain energy-scalable stress monitoring models. We further propose a self-aware stress monitoring method that intelligently switches among the energy-scalable models, reducing energy consumption. Results: Using CAFS methodology on experimental data and simulation, we reduce the energy-cost of the stress model designed without energy constraints up to 94.37%. We obtain 90.98% and 95.74% as the best accuracy and confidence values, respectively , on unseen data, outperforming state-of-the-art studies. Analyzing our interpretable and energy-scalable models, we showed that simple models using only heart rate (HR) or skin conductance level (SCL), confidently predict acute stress for HR > 93.30BP M and non-stress for SCL < 6.42µS, but, outside these values, a multimodal model using respiration and pulse wave's features is needed for confident classification. Our self-aware acute stress monitoring proposal saves 10x energy and provides 88.72% of accuracy on unseen data. Conclusion: We propose a comprehensive solution for the cost-aware acute stress monitoring design addressing the problem of selecting an optimized feature subset considering their cost-dependency and cost-constraints. Significant: Our design framework enables long-term and confident acute stress monitoring on wearable devices.

2021

Multi-Centroid Hyperdimensional Computing Approach for Epileptic Seizure Detection

David Atienza Alonso, Una Pale, Tomas Teijeiro Campo

Long-term monitoring of patients with epilepsy presents a challenging problem from the engineering perspective of real-time detection and wearable devices design. It requires new solutions that allow continuous unobstructed monitoring and reliable detection and prediction of seizures. A high variability in the electroencephalogram (EEG) patterns exists among people, brain states, and time instances during seizures, but also during non-seizure periods. This makes epileptic seizure detection very challenging, especially if data is grouped under only seizure (ictal) and non-seizure (inter-ictal) labels. Hyperdimensional (HD) computing, a novel machine learning approach, comes in as a promising tool. However, it has certain limitations when the data shows a high intra-class variability. Therefore, in this work, we propose a novel semi-supervised learning approach based on a multi-centroid HD computing. The multi-centroid approach allows to have several prototype vectors representing seizure and non-seizure states, which leads to significantly improved performance when compared to a simple single-centroid HD model. Further, real-life data imbalance poses an additional challenge and the performance reported on balanced subsets of data is likely to be overestimated. Thus, we test our multi-centroid approach with three different dataset balancing scenarios, showing that performance improvement is higher for the less balanced dataset. More specifically, up to 14% improvement is achieved on an unbalanced test set with 10 times more non-seizure than seizure data. At the same time, the total number of sub-classes is not significantly increased compared to the balanced dataset. Thus, the proposed multi-centroid approach can be an important element in achieving a high performance of epilepsy detection with real-life data balance or during online learning, where seizures are infrequent.

2022