Motion capture | EPFL Graph Search

Motion capture (sometimes referred as mo-cap or mocap, for short) is the process of recording the movement of objects or people. It is used in military, entertainment, sports, medical applications, and for validation of computer vision and robots. In filmmaking and video game development, it refers to recording actions of human actors and using that information to animate digital character models in 2D or 3D computer animation. When it includes face and fingers or captures subtle expressions, it is often referred to as performance capture. In many fields, motion capture is sometimes called motion tracking, but in filmmaking and games, motion tracking usually refers more to match moving. In motion capture sessions, movements of one or more actors are sampled many times per second. Whereas early techniques used , often the purpose of motion capture is to record only the movements of the actor, not their visual appearance. This animation data is mapped to a 3D model so that the model

Using biomechanical constraints to improve video-based motion capture

Lorna Herda

In motion capture applications whose aim is to recover human body postures from various input, the high dimensionality of the problem makes it desirable to reduce the size of the search-space by eliminating a priori impossible configurations. This can be carried out by constraining the posture recovery process in various ways. Most recent work in this area has focused on applying camera viewpoint-related constraints to eliminate erroneous solutions. When camera calibration parameters are available, they provide an extremely efficient tool for disambiguating not only posture estimation, but also 3D reconstruction and data segmentation. Increased robustness is indeed to be gained from enforcing such constraints, which we prove in the context of an optical motion capture framework. Our contribution in this respect resides in having applied such constraints consistently to each main step involved in a motion capture process, namely marker reconstruction and segmentation, followed by posture recovery. These steps are made inter-dependent, where each one constrains the other. A more application-independent approach is to encode constraints directly within the human body model, such as limits on the rotational joints. This being an almost unexplored research subject, our efforts were mainly directed at determining a new method for measuring, representing and applying such joint limits. To the present day, the few existing range of motion boundary representations present severe drawbacks that call for an alternative formulation. The joint limits paradigm we propose not only overcomes these drawbacks, but also allows to capture intra- and inter-joint rotation dependencies, these being essential to realistic joint motion representation. The range of motion boundary is defined by an implicit surface, its analytical expression enabling us to readily establish whether a given joint rotation is valid or not. Furthermore, its continuous and differentiable nature provides us with a means of elegantly incorporating such a constraint within an optimisation process for posture recovery. Applying constrained optimisation to our body model and stereo data extracted from video sequence, we demonstrate the clearly resulting decrease in posture estimation errors. As a bonus, we have integrated our joint limits representation in character animation packages to show how motion can be naturally constrained in this manner.

EPFL2004

Machine Learning for Musculoskeletal Modeling of Upper Extremity

Rahul Sharma

We propose a novel machine learning (ML)-driven methodology to estimate biomechanical variables of interest traditionally obtained from upper-extremity musculoskeletal (MSK) modeling. MSK models facilitate personalized modeling, perform "what-if" analyses, and potentially enhance clinical decision-making. In certain settings, MSK models are driven by inertial motion capture (IMC) data. IMC systems are portable, user-friendly, and relatively affordable as well as provide additional biomechanical information. However, MSK models can be computationally expensive, often require extensive data, and can be prohibitively slow in making real-time predictions. Our ML method- involving a rigorous hyperparameters search-predicts kinematic and kinetic biomechanical information associated with human upper-extremity movements solely using IMC input data, thereby bypassing MSK models. The scaled cadaver-based MSK model was based on the Dutch Shoulder Model and the spine model implemented in the AnyBody Managed Model Repository. We employ neural networks (NNs), which are trained on IMC data obtained from five nondisabled subjects in subject- exposed (SE) settings (at least a trial from all subjects is used in training) and subject- naive (SN) settings (no information from test subjects is used in training). We compare the predictions of our ML model with that of an MSK model and find an excellent agreement in SE settings (average Pearson's r = 0.96, normalized RMSE (NRMSE) = 0.23) and strong correspondence in SN settings (average r = 0.89, NRMSE = 0.45). The linear model performed poorly for SN settings, which motivated a more complex ML model. Our findings suggest that an ML-based approach is highly viable for estimating upper-extremity biomechanical information solely from IMC data.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives

Jessy Lauer, Mackenzie Mathis, Alexander Mathis, Steffen Schneider

Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced our ability to predict posture directly from videos, which has quickly impacted neuroscience and biology more broadly. In this primer, we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.

2020

Using biomechanical constraints to improve video-based motion capture

Lorna Herda

EPFL2004