Systematic Evaluation Methodology and Performance Metrics for Haptic Interfaces

The purpose of evaluation procedures for haptic interfaces is to achieve both qualitative and quantitative statements on haptic rendering realism and performance. Since haptics technology is being increasingly used in computer games, surgical simulators, mobile phones etc., there is a need for defining standards for haptic applications. This study aims at meeting this need by establishing a norm for evaluation of haptic interfaces and by identifying significant benchmark metrics. Towards this end, a combined physical and psychophysical experimental methodology is proposed in this work. First, the existing physical performance measures and device characterization techniques were investigated and described in an illustrative way. The physical characterization methods were demonstrated on a novel two degrees-of-freedom haptic interface for colonoscopy simulation. Second, a wide range of human psychophysical experiments were reviewed and the appropriate ones were applied to haptic interactions. The psychophysical experiments were unified as a systematic and complete evaluation method for haptic interfaces. Seven psychophysical tests were derived and implemented for three commercial force-feedback devices. Experimental user studies were carried out and applicability of the tests to a tactile feedback device was also investigated. The physical evaluation resulted in a detailed tutorial-like guideline for device characterization. Testing conditions and methods were described to allow performing identical tests with other devices. The psychophysical evaluation resulted in novel benchmark metrics for haptic interfaces. The performance metrics are expressed in terms of information transfer (bits) and sensory thresholds, which are indeed device specific metrics. They are not only related to the basic physical characteristics of haptic interfaces, but also easier to be linked with the rendering quality perceived by the user. The results prove that the metrics are suitable to compare force-feedback devices. The metrics reveal the basic characteristics of the devices which are not easily derived from the given specifications. The results were also compared with those from human psychophysical experiments with real objects. Significant differences clearly demonstrate the limits of the haptic interfaces. In conclusion, the generic methodology proposed in this work enables users to evaluate the suitability of a haptic interface for a specific purpose, to characterize and compare devices quantitatively and to identify possible improvement strategies in the design of the system.