Towards real-world biomechanical analysis of performance and functional capacity using wearable sensors

Today, wearable heart rate monitors are an integral part of runners' training sessions, with heart rate data routinely used to assess effort intensity and stress on the body. As athletes translate their physical capacity into performance on the field through their movement, biomechanical assessment can provide valuable information that complements physiological assessment. However, the potential of using biomechanical information in the evaluation of training sessions and standardized tests in practice remains largely untapped, partly because the assessment devices remain cumbersome to use and often require long post-processing, as well as programming skills. The proposed work aims to realize this potential by developing field methods for performance and capacity evaluation using portable inertial measurement units (IMUs) and Global Navigation Satellite System (GNSS) receivers.Running performance can be characterized by the ability to maintain appropriate running technique despite fatigue, while keeping the effort intensity prescribed by the coach, or planned as a pacing strategy. In this work, a systematic review was conducted to examine and synthesize the results of fatigue protocols in running, fol-lowed by continuous measurements during a competition to confirm the trends obtained from the review with data from the field and to measure the changes in running technique due to fatigue. In addition, models were developed to accurately estimate running power using foot-worn IMU over a range of speeds and inclines and validated using gold standard methods in the laboratory, to better characterize running intensity. The second part of this work consisted of investigating the ability of IMUs and GNSS to improve the evaluation of athletes in standardized tests, referred to as their functional capacity. Functional capacity is typically used by coaches to develop appropriate training loads for athletes. This work presented validated methods to instrument common functional tests with wearable sensors to measure the speed, agility, and endurance of athletes in the field. In addition, these methods enable the extraction and a deeper analysis of relevant biomechanical parameters that contribute to the measured capacity and help the sporting staff understand athletes' strengths and weaknesses in detail. All of the research conducted and methods developed in this work are based on various combinations of a minimal body-worn sensor setup with foot-worn IMUs and a single trunk-worn IMU-GNSS unit. The signal processing algorithms and models developed in this work allow the recorded signals to be translated into easily interpretable and actionable information. Based on this information, coaches and physical therapists can develop customized training programs that target the relevant parameters. The proposed sensor setups and methods have been used and validated in a variety of situations, such as pre-season testing of a professional soccer team, training sessions of elite sprinters, the Lausanne half-marathon race, etc., highlighting their potential for real-world application. I believe that this work will help pave the way towards a deeper understanding of the biomechanical contributions to performance in running and provide new tools for the development of personalized training and rehabilitation programs, with the goal of optimizing positive adaptation to training stimuli, thereby improving performance and reducing the incidence of injury.

Conceptual modeling of multimedia databases

Oleksandr Drutskyy

The gap between the semantic content of multimedia data and its underlying physical representation is one of the main problems in the modern multimedia research in general, and, in particular, in the field of multimedia database modeling. We believe that one of the principal reasons of this problem is the attempt to conceptually represent multimedia data in a way, which is similar to its low-level representation by applications dealing with encoding standards, feature-based multimedia analysis, etc. In our opinion, such conceptual representation of multimedia contributes to the semantic gap by separating the representation of multimedia information from the representation of the universe of discourse of an application, to which the multimedia information pertains. In this research work we address the problem of conceptual modeling of multimedia data in a way to deal with the above-mentioned limitations. First, we introduce two different paradigms of conceptual understanding of the essence of multimedia data, namely: multimedia as data and multimedia as metadata. The multimedia as data paradigm, which views multimedia data as the subject of modeling in its own right, is inherent to so-called multimedia-centric applications, where multimedia information itself represents the main part of the universe of discourse. The examples of such kind of applications are digital photo collections or digital movie archives. On the other hand, the multimedia as metadata paradigm, which is inherent to so-called multimedia-enhanced applications, views multimedia data as just another (optional) source of information about whatever universe of discourse that the application pertains to. An example of a multimedia-enhanced application is a human-resource database augmented with employee photos. Here the universe of discourse is the totality of company employees, while their photos simply represent an additional (possibly optional) kind of information describing the universe of discourse. The multimedia conceptual modeling approach that we present in this work allows addressing multimedia-centric applications, as well as, in particular, multimedia-enhanced applications. The model that we propose builds upon MADS (Modeling Application Data with Spatio-temporal features), which is a rich conceptual model defined in our laboratory, and which is, in particular, characterized by structural completeness, spatio-temporal modeling capabilities, and multirepresentation support. The proposed multimedia model is provided in the form of a new modeling dimension of MADS, whose orthogonality principle allows to integrate the new multimedia modeling dimension with already existing modeling features of MADS. The following multimedia modeling constructs are provided: multimedia datatypes, simple and complex representational constraints (relationships), a multimedia partitioning mechanism, and multimedia multirepresentation features. Following the description of our conceptual multimedia modeling approach based on MADS, we present the peculiarities of logical multimedia modeling and of conceptual-to-logical inter-layer transformations. We provide a set of mapping guidelines intended to help the schema designer in coming up with rich logical multimedia document representations of the application domain, which conform with the conceptual multimedia schema. The practical interest of our research is illustrated by a mock-up application, which has been developed to support the theoretical ideas described in this work. In particular, we show how the abstract conceptual set-based representations of multimedia data elements, as well as simple and complex multimedia representational relationships can be implemented using Oracle DBMS.

EPFL2009

Enactive robot vision

Mototaka Suzuki

The complexity of today's autonomous robots poses a major challenge for Artificial Intelligence. These robots are equipped with sophisticated sensors and mechanical abilities that allow them to enter our homes and interact with humans. For example, today's robots are almost all equipped with vision and several of them can move over rough terrain with wheels or legs. The methods developed so far in Artificial Intelligence, however, are not yet ready to cope with the complexity of the information gathered through the robot sensors and the need for rapid action in partially unknown and dynamic environments. In this thesis, I will argue that the apparent complexity of the environment and of the robot brain can be significantly simplified if perception, behavior, and learning are allowed to co-develop on the same time scale. In doing so, robots become sensitive to, and actively exploit, characteristics of the environment that they can tackle within their own computational and physical constraints. This line of work is grounded on philosophical and psychological research showing that perception is an active process mediated by behavior. However, computational models of active vision are very rare and often rely on architectures that are preprogrammed to detect certain characteristics of the environment. Previous work have shown that complex visual tasks, such as position and size invariant shape recognition as well as navigation, can be tackled with remarkably simple neural architectures generated by a coevolutionary process of active vision and feature selection. Behavioral machines equipped with primitive vision systems and direct pathways between visual and motor neurons were evolved while freely interacting with their environments. I proceed further on this line of investigation and describe the application of this methodology in three situations, namely car driving with an omnidirectional camera, goal-oriented navigation of a humanoid robot, and cooperative tasks by two agents. I will show that these systems develop sensitivity to a number of oriented, retinotopic, visual features – oriented edges, corners, height – and a behavioral repertoire to locate, bring, and keep these features in sensitive regions of the vision system that allow them to accomplish their goals. In a second set of experiments, I will show that active vision can be exploited by the robot to perform anticipatory exploration of the environment in a task that requires landmark-based navigation. Evolved robots exploit an internal expectation system that makes use of active exploration to check for expected events in the environment. I will describe a third set of experiments where, in addition to an evolutionary process, the visual system of the robot can develop receptive fields by means of unsupervised Hebbian learning and show that these receptive fields are significantly affected by the behavior of the system and differ from those predicted by most computational models of visual cortex. Finally, I will show that these robots replicate the performance deficiencies observed in experiments of motor deprivation with kitten when they are exposed to the same type of motor deprivations. Furthermore, the analyses of our robot brains suggest an explanation for the deficiencies observed in kitten that have not yet been fully understood.

EPFL2007

Assessment of Foot Signature Using Wearable Sensors for Clinical Gait Analysis and Real-Time Activity Recognition

Benoît Mariani

Locomotion is one of the most important abilities of humans. Actually, gait locomotion provides mobility, and symbolizes freedom and independence. However, gait can be affected by several pathologies, due to aging, neurodegenerative disease, or trauma. The evaluation and treatment of mobility diseases thus requires clinical gait assessment, which is commonly done by using either qualitative analysis based on subjective observations and questionnaires, or expensive analysis established in complex motion laboratories settings. This thesis presents a new wearable system and algorithmic methods for gait assessment in natural conditions, addressing the limitations of existing methods. The proposed system provides quantitative assessment of gait performance through simple and precise outcome measures. The system includes wireless inertial sensors worn on the foot, that record data unobtrusively over long periods of time without interfering with subject's walking. Signal processing algorithms are presented for the automatic calibration and online virtual alignment of sensor signals, the detection of temporal parameters and gait phases, and the estimation of 3D foot kinematics during gait based on fusion methods and biomechanical assumptions. The resulting 3D foot trajectory during one gait cycle is defined as Foot Signature, by analogy with hand-written signature. Spatio-temporal parameters of interest in clinical assessment are derived from foot signature, including commonly parameters, such as stride velocity and gait cycle time, as well as original parameters describing inner-stance phases of gait, foot clearance, and turning. Algorithms based on expert and machine learning methods have been also adapted and implemented in real-time to provide input features to recognize locomotion activities including level walking, stairs, and ramp locomotion. Technical validation of the presented methods against gold standard systems was carried out using experimental protocols on subjects with normal and abnormal gait. Temporal aspects and quantitative estimation of foot-flat were evaluated against pressure insoles in subjects with ankle treatments during long-term gait. Furthermore, spatial parameters and foot clearance were compared in young and elderly persons to data obtained from an optical motion capture system during forward gait trials at various speeds. Finally, turning was evaluated in children with cerebral palsy and people with Parkinson's disease against optical motion capture data captured during timed up and go and figure-of-8 tests. Overall, the results demonstrated that the presently proposed system and methods were precise and accurate, and showed agreement with reference systems as well as with clinical evaluations of subjects' mobility disease using classical scores. Currently, no other methods based on wearable sensors have been validated with such precision to measure foot signature and subsequent parameters during unconstrained walking. Finally, we have used the proposed system in a large-scale clinical application involving more than 1800 subjects from age 7 to 77. This analysis provides reference data of common and original gait parameters, as well as their relationship with walking speed, and allows comparisons between different groups of subjects with normal and abnormal gait. Since the presented methods can be used with any foot-worn inertial sensors, or even combined with other systems, we believe our work to open the door to objective and quantitative routine gait evaluations in clinical settings for supporting diagnosis. Furthermore, the present studies have high potential for further research related to rehabilitation based on real-time devices, the investigation of new parameters' significance and their association with various mobility diseases, as well as for the evaluation of clinical interventions.

EPFL2012