A Virtual Keyboard Based on True-3D Optical Ranging

Edoardo Charbon, Huan Du, Félix Lustenberger
2005
Article de conférence

Résumé

In this paper, a complete system is presented which mimics a QWERTY keyboard on an arbitrary surface. The system consists of a pattern projector and a true-3D range camera for detecting the typing events. We exploit depth information acquired with the 3D range camera and detect the hand region using a pre-computed reference frame. The fingertips are found by analyzing the hands' contour and fitting the depth curve with different feature models. To detect a keystroke, we analyze the feature of the depth curve and map it back to a global coordinate system to find which key was pressed. These steps are fully automated and do not require human intervention. The system can be used in any application requiring zero form factor and minimized or no contact with a medium, as in a large number of cases in human-to-computer interaction, virtual reality, game control, 3D designs, etc.

Official source

https://infoscience.epfl.ch/record/87512?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Crowds for interactive virtual environments

Branislav Ulicny

Crowds are part of our everyday experience; nevertheless, in virtual worlds they are still relatively rare. In this thesis we explore the challenges of bringing crowds into interactive virtual environments and the possible applications of real-time interactive crowds, for example, in the domains of virtual heritage and training systems. We address several topics including real-time generation of higher-level behaviors, lower-level motion control for many characters, management of variety, integration of crowds into virtual environments, interaction with virtual crowds and authoring of scenarios. In comparison with the other crowd modeling approaches, we focus on more complex behaviors of many agents in dynamic environments with possible user interaction. In our system a crowd is modeled as a collection of individuals which react to the environment, to other agents and to real human participants of the simulation and can have very different behaviors, both for one agent in different situations, and for many agents in the same situation. We emphasize the importance of variety as one of the most distinguishing features of the crowd simulations; we introduce the notion of levels-of-variety. We address as well the question of how to author crowd scenes in an efficient way. We introduce a novel approach to create complex scenes involving thousands of animated individuals in a simple and intuitive way. By employing a brush metaphor, analogous to the tools used in image manipulation programs, we can distribute, modify and control crowd members in real-time with immediate visual feedback. We demonstrate the validity of our behavior simulation approach with several case studies, where we use our crowd system to manage crowd behaviour in a virtual reality training system and virtual heritage reconstruction. The potential of our authoring approach is shown by authoring a scenario of a virtual audience in a theater and a scenario of a pedestrian crowd in a city.

EPFL2005

Chargement

Conferring human action recognition skills to life-like agents

Ronan Boulic, Daniel Thalmann

Most of today's virtual environments are populated with some kind of autonomous life-like agents. Such agents follow a preprogrammed sequence of behaviours that excludes the user as a participating entity in the virtual society. In order to make inhabited virtual reality an attractive place for information exchange and social interaction, we need to equip the autonomous agents with some perception and interpretation skills. We present one skill: human action recognition. By opposition to human-computer interfaces that focus on speech or hand gestures, we propose a full-body integration of the user. We present a model of human actions along with a real time recognition system. To cover the bilateral aspect in human-computer interfaces, we also discuss some action response issues. In particular, we describe a motion management library that solves animation continuity and mixing problems. Finally, we illustrate our system with two examples and discuss what we have learned

1999

Chargement

In this work, we propose new ways to employ 3D-ranging systems in advanced human-computer interfaces and show that they can be used for precise hand-tracking systems aiming at virtual keyboard or mouse applications. We first implement a Structured Light (SL) based 3D-ranging system as the testbed for acquiring depth information. The ranging system uses an off-the-shelf projector combined with an industrial color camera, where the projector projects well-designed line patterns onto the scene and the camera captures the reflected light from the scene and the target. The depth information is recovered from the deformed line pattern and the pre-calibrated projector-camera system. With this system we can capture dense depth maps with moderate resolution and high frame rate as the simulated input to the hand-tracking system. We then explore an approach to 3D hand matching that combines a detailed 3D hand model (skeletal + polygonal model) with a Physical-based Model Fitting technique (PMF). Virtual forces are assigned between the depth measurements and the polygonal vertices of the model. The matching problem is formulated as the minimization of the distances between the measurement and the model surface with the guidance of virtual forces. This can employ 3D information simply and directly onto the parameter space, and yield good results under challenging conditions. To demonstrate the effectiveness of the approach we show that we can track hand motions such as typing or grasping frame by frame. The major limitation of this method is that it requires a good initialization of the hand model and the computation time to achieve convergence with good fitting results is quite long. We therefore investigated more complex hand models and feature-based matching techniques. A three-level hand model (skeletal + primitive + polygonal model) is proposed for the Multi-Level Feature Matching approach (MLFM). The matching is carried out from the bottom to top levels of different hand presentations, and distinctive features for each level extracted from both the measurements and the model are matched with Scaled Conjugate Gradient method (SCG). The matching result from lower level is used as the initial state of the model parameters for higher level matching. This hierarchical structure speeds up the optimization process and enhances the precision of the matching result. We demonstrate our approach with the same hand motion as in the previous method and compare the performance of these two methods. We show that the MLFM method is faster and more robust to hand occlusions than the PMF method. Finally, we present a more powerful approach based on the Back-Constrained Gaussian Process Latent Variable Model (BC-GPLVM) that import prior information of categorized hand motion such as typing. BC-GPLVM provides a low-dimensional embedding of hand motion data, with a density function that gives higher probability to gestures and motions close to normal typing motion. This model is learned from a small set of clusters of vast captured motion data. The prior model is then combined with the previous multi-level models and features to generate a uniform cost function for optimization. We show the tracking result to demonstrate that this method can be used for human-computer interfaces requiring precise 3D hand tracking.

EPFL2008

Chargement

EPFL2008

Crowds for interactive virtual environments

Branislav Ulicny

EPFL2005