Precise and Responsive Performance Animation for Embodied Immersive Interactions

The recent advances in immersive display technologies offer a unique opportunity for providing intuitive interaction techniques easing the user involvement in the Virtual Environment. This trend underlines the great potential of Performance Animation especially for applications with real-time interactions. Capturing the motion of the performer and mapping it onto an avatar are not new problems, but still have several research challenges to be addressed. Ensuring the highest fidelity of the reconstructed postures while maintaining the responsiveness of the interaction is a difficult problem, in particular, when embodying an avatar with different body size and proportions. In line with these requirements, we introduce several posture tracking, reconstruction, representation and retargetting techniques. First, we introduce a semi-automated calibration technique to precisely track the performer's body. Besides computing a digital representation of the performer's skeletal structure, our technique registers the orientation of the body parts and identifies an approximation of the body surface. We propose then a novel parametrization of human limbs that addresses the ill-conditioned cases of analytical inverse kinematics algorithms. We also integrate it within a Jacobian based linearized inverse kinematics framework for obtaining faster convergence. We exploit a low-cost posture reconstruction technique for full-body real time control of an avatar with the same body size and proportions of the performer. We present its usability in two types of applications: Virtual Reality based rehabilitation and immersive motion analysis. Finally, we propose an egocentric normalization of the body-part relative distances to preserve the consistency of self-contacts for a large variety of human-like target characters. Egocentric coordinates are character independent and encode the whole posture space, i.e., it ensures the continuity of the motion with and without self-contacts. We can transfer classes of complex postures involving multiple interacting limb segments by preserving the limb relative spatial order without depending on temporal coherence.