A Virtual Keyboard System based on Multi-Level Feature Matching

In this paper a Multi-Level Feature Matching (MLFM) method is presented for 3D hand posture reconstruction of a virtual keyboard system. The human hand is modeled with a mixture of different levels of detail, from skeletal to polygonal surface representation. Different types of features are extracted and paired with the corresponding model. The matching is performed in a bottom-up order by SCG optimization with respect to the state vector of motion parameters. The low level of matching provide initial guess to the high level of matching, refining the precise position of the hand hierarchically. The matching results show that this method is effective for tracking human hand typing motion, even with noisy 3D depth map reconstruction and roughly detected fingertips. Examples of applications include virtual reality, gaming, 3D design, etc.