Real-Time EEG-Based Cognitive Workload Monitoring on Wearable Devices

Objective: Cognitive workload monitoring (CWM)can enhance human-machine interaction by supporting task execution assistance considering the operator’s cognitive state. Therefore, we propose a machine learning design methodology and a data processing strategy to enable CWM on resource-constrained wearable devices. Methods: Our CWM solution is built upon edge computing on a simple wearable system, with only four peripheral channels of electroencephalography (EEG). We assess our solution on experimental data from 24 volunteers. Moreover, to overcome the system’s memory constraints, we adopt an optimization strategy for model size reduction and a multi-batch data processing scheme for optimizing RAM memory footprint. Finally, we implement our data processing strategy on a state-of-the-art wearable platform and assess its execution and system battery life. Results: We achieve an accuracy of 74.5% and a 74.0% geometric mean between sensitivity and specificity for cwm classification on unseen data. Besides, the proposed model optimization strategy generates a 27.5x smaller model compared to the one generated with default parameters, and the multi-batch data processing scheme reduces RAM memory footprint by 14x compared to a single batch data processing. Finally, our algorithm uses only 1.28% of the available processing time, thus, allowing our system to achieve 28.5 hours of battery life. Conclusion: We provide a reliable and optimized CWM solution using wearable devices, enabling more than a day of operation on a single battery charge. Significance: The proposed methodology enables real-time data processing on resource-constrained devices and supports real-time wearable monitoring based on EEG for applications as CWM in human-machine interaction.

New Measurement Method in Drug Sensing by Direct Total-Charge Detection in Voltammetry

Simone Aiassa, Sandro Carrara, Danilo Demarchi, José David González Martínez

Electrochemical biosensors are promoting point-of-care and wearable instrumentation due to their high versatility in measuring human metabolites. There is a considerable number of biological compounds that can be detected and measured through voltammetry based techniques. Voltmmetry some times requires peak identification and quantification that are non-trivial to be efficiently implemented by automatic instrumentation. To overcome the complexity of automatic peak estimation, we propose here an instrumentation circuit for edge-computing in pharmacology relying on an entirely novel measurement method via TotalCharge Detection in Cyclic voltammetry (TCDC). Namely, our TCDC method innovatively applies the coulometry measurement to the well-established voltammetry procedure. The proposed instrumentation accumulates the total charge exchanged in the faradaic process, exploiting a Nagaraj integrator as charge suppressor to fit the application-specific constraints. The work shows accurate simulations of the TCDC circuit on a set of experimental measures, acquired on paracetamol as benchmark drug. The proposed measurement technique and the developed circuit are compared to the peak detection method usually adopted in literature. The results demonstrate that the proposed system is a perfect trade-off between the doubled limit-of-detection and a tenfold reduction in measurement errors. At the same time, we eliminate any need for data oversampling and processing, promoting the TCDC as an efficient new measurement method for point-of-care and wearable monitoring of biological compounds.

IEEE2020

Wearable and Self-Aware Machine Learning System for Online Cognitive Workload Monitoring and Drone Control

The use of robots in search and rescue is gaining particular interest, but singular skills are required to ensure efficient deployments in real missions. To face this problem, there is a need to develop more intuitive control interfaces. Moreover, to ensure high performance during high cognitive demanding tasks, such as in search and rescue missions, there is a need to combine both humanâs and robotâs skills. Respectively, humans and robots can adapt to new situations and optimize the execution of repetitive tasks. In this regard, novel share-control techniques have been developed to adapt the human-robot interaction, but to dynamically adapt this interaction, the information about the human state is missing. To address these problems, I first developed a novel wearable system that enhances the control of drones providing a more intuitive flying experience. As shown in chapter 2, this wearable system tracks the upper body movements and translates them into commands for a drone. The system has been tested with a simulator and demonstrated for the teleoperation of a real drone. Moreover, to ensure enduring operations, I proposed a method that drastically reduces communication, and consequently, improves energy efficiency by 11.9%. Second, in chapter 3, I presented a machine-learning approach for monitoring the cognitive workload level of a drone operator involved in search and rescue missions. My model combines the information of different features extracted from physiological signals, such as respiratory activity, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a non-invasive way. To reduce both subject and day inter-variability of the signals, I explored different feature normalization techniques. Moreover, I adjusted the learning method for support vector machines to allow subject-specific optimizations. On a test set acquired from 34 volunteers, the proposed model distinguished between low and high cognitive workloads with an average accuracy of 87.3% and 91.2%, while controlling a drone simulator using both a traditional controller and the proposed FlyJacket design, respectively. Third, in chapter 4, I presented the integration of the method developed for cognitive workload monitoring, on a new single wearable embedded system, that also integrates the proposed drone controller design. On the hardware side, it includes a multi-channel physiological signals acquisition and a low-power processing platform that is suited for cognitive workload monitoring. On the software side, the proposed system includes novel energy-aware bio-signal processing and embedded machine learning methods. Moreover, to exploit the trade-offs between the required accuracy of the available energy of the system, I presented a new application of the concept of a scalable machine-learning method with different power-saving levels. Results showed that the proposed self-aware approach yields an increase of 78% of the battery lifetime without really affecting the classification accuracy. The proposed system, comprising a drone controller integrating a unit for cognitive workload monitoring, lays the foundations for the development of new-generation human-robot interfaces. With the information about the human state, we can close the loop of traditional share-control techniques, which will be able to dynamically adapt the level of interaction with semi-autonomous machines based on the need for the operator.

EPFL2020

Assessment of Foot Signature Using Wearable Sensors for Clinical Gait Analysis and Real-Time Activity Recognition

Benoît Mariani

Locomotion is one of the most important abilities of humans. Actually, gait locomotion provides mobility, and symbolizes freedom and independence. However, gait can be affected by several pathologies, due to aging, neurodegenerative disease, or trauma. The evaluation and treatment of mobility diseases thus requires clinical gait assessment, which is commonly done by using either qualitative analysis based on subjective observations and questionnaires, or expensive analysis established in complex motion laboratories settings. This thesis presents a new wearable system and algorithmic methods for gait assessment in natural conditions, addressing the limitations of existing methods. The proposed system provides quantitative assessment of gait performance through simple and precise outcome measures. The system includes wireless inertial sensors worn on the foot, that record data unobtrusively over long periods of time without interfering with subject's walking. Signal processing algorithms are presented for the automatic calibration and online virtual alignment of sensor signals, the detection of temporal parameters and gait phases, and the estimation of 3D foot kinematics during gait based on fusion methods and biomechanical assumptions. The resulting 3D foot trajectory during one gait cycle is defined as Foot Signature, by analogy with hand-written signature. Spatio-temporal parameters of interest in clinical assessment are derived from foot signature, including commonly parameters, such as stride velocity and gait cycle time, as well as original parameters describing inner-stance phases of gait, foot clearance, and turning. Algorithms based on expert and machine learning methods have been also adapted and implemented in real-time to provide input features to recognize locomotion activities including level walking, stairs, and ramp locomotion. Technical validation of the presented methods against gold standard systems was carried out using experimental protocols on subjects with normal and abnormal gait. Temporal aspects and quantitative estimation of foot-flat were evaluated against pressure insoles in subjects with ankle treatments during long-term gait. Furthermore, spatial parameters and foot clearance were compared in young and elderly persons to data obtained from an optical motion capture system during forward gait trials at various speeds. Finally, turning was evaluated in children with cerebral palsy and people with Parkinson's disease against optical motion capture data captured during timed up and go and figure-of-8 tests. Overall, the results demonstrated that the presently proposed system and methods were precise and accurate, and showed agreement with reference systems as well as with clinical evaluations of subjects' mobility disease using classical scores. Currently, no other methods based on wearable sensors have been validated with such precision to measure foot signature and subsequent parameters during unconstrained walking. Finally, we have used the proposed system in a large-scale clinical application involving more than 1800 subjects from age 7 to 77. This analysis provides reference data of common and original gait parameters, as well as their relationship with walking speed, and allows comparisons between different groups of subjects with normal and abnormal gait. Since the presented methods can be used with any foot-worn inertial sensors, or even combined with other systems, we believe our work to open the door to objective and quantitative routine gait evaluations in clinical settings for supporting diagnosis. Furthermore, the present studies have high potential for further research related to rehabilitation based on real-time devices, the investigation of new parameters' significance and their association with various mobility diseases, as well as for the evaluation of clinical interventions.

EPFL2012

Assessment of Foot Signature Using Wearable Sensors for Clinical Gait Analysis and Real-Time Activity Recognition

Benoît Mariani

EPFL2012

Wearable and Self-Aware Machine Learning System for Online Cognitive Workload Monitoring and Drone Control

EPFL2020