Precise and accurate image-based measurements of optical material properties

The appearance of objects is governed by how they reflect, transmit and absorb the light they receive. That, in turn, depends on the material's internal structure, surface structure, and viewing and illumination directions. Changes in those characteristics can produce dramatic shifts in a material's appearance.

In this thesis, we explore how the optical properties can be estimated accurately using computational imaging. Current solutions are based on using instruments that are on the expensive side, such as spectrophotometers, profilometers and goniophotometers; hence, we explore the use of more affordable equipment, such as cameras. We start with the reflected light, and how the surface structure and roughness can be measured with commodity hardware such as projectors and cameras. We continue to the transmitted light, where we study methods for measuring hemispherical and directional transmittance.

Systems based on the projection of structured light can be used for high-precision depth estimation. These methods provide non-contact means of depth estimation by using a calibrated camera-projector pair. Calibration of such systems is a cumbersome and time-consuming task. We, thus, present a novel method that allows for simultaneous geometric and radiometric calibration of a projector-camera pair. We leverage the different colorimetric properties of the printed and projected patterns to perform the geometric calibration, and perform the radiometric calibration of the projector with the information contained inside the projected squares. We show that our method performs on par with current approaches that all require separate geometric and radiometric calibration. We then apply the algorithm to calibrate a structured light system with the goal of surface structure estimation of nearly planar surfaces. The surface structure is then used to calculate their macroscopic surface roughness. Over a set of samples, our results are comparable to those from a high-grade commercial profilometer.

The accurate measurement of transmittance properties of materials is essential in the printing and display industries to ensure accurate color reproduction. We explore different measurement geometries for total transmittance, and show that the measurements are highly affected by the geometry used, since certain geometries can introduce a measurement bias. We build a flexible custom setup that can simulate these geometries, and evaluate it qualitatively and quantitatively. We also compare its measurements to those of widely used commercial solutions, and show that significant differences exist over our test set. These findings stress the importance of including the measurement geometry when reporting total transmittance.

The BSDF is of great importance in many applications, from rendering and visual special effects to architectural modelling of illumination. We propose a novel design for the measurement of BSDF, more specifically the BTDF. It is based on radial catadioptric imaging, where we illuminate the measured sample with a collimated beam of light, the emerging light on the other side of the sample is collected by a diffusing cylinder, and a camera registers the steady-state radiance. After inverting the light transport inside the cylinder, we can compute the sample's BTDF. In simulations, our proposed design achieves results that are well aligned with the ground truth, for both smooth and high-frequency BTDFs that feature sharp intensity changes.

Application of 3D range camera in virtual human-computer interfaces

Huan Du

In this work, we propose new ways to employ 3D-ranging systems in advanced human-computer interfaces and show that they can be used for precise hand-tracking systems aiming at virtual keyboard or mouse applications. We first implement a Structured Light (SL) based 3D-ranging system as the testbed for acquiring depth information. The ranging system uses an off-the-shelf projector combined with an industrial color camera, where the projector projects well-designed line patterns onto the scene and the camera captures the reflected light from the scene and the target. The depth information is recovered from the deformed line pattern and the pre-calibrated projector-camera system. With this system we can capture dense depth maps with moderate resolution and high frame rate as the simulated input to the hand-tracking system. We then explore an approach to 3D hand matching that combines a detailed 3D hand model (skeletal + polygonal model) with a Physical-based Model Fitting technique (PMF). Virtual forces are assigned between the depth measurements and the polygonal vertices of the model. The matching problem is formulated as the minimization of the distances between the measurement and the model surface with the guidance of virtual forces. This can employ 3D information simply and directly onto the parameter space, and yield good results under challenging conditions. To demonstrate the effectiveness of the approach we show that we can track hand motions such as typing or grasping frame by frame. The major limitation of this method is that it requires a good initialization of the hand model and the computation time to achieve convergence with good fitting results is quite long. We therefore investigated more complex hand models and feature-based matching techniques. A three-level hand model (skeletal + primitive + polygonal model) is proposed for the Multi-Level Feature Matching approach (MLFM). The matching is carried out from the bottom to top levels of different hand presentations, and distinctive features for each level extracted from both the measurements and the model are matched with Scaled Conjugate Gradient method (SCG). The matching result from lower level is used as the initial state of the model parameters for higher level matching. This hierarchical structure speeds up the optimization process and enhances the precision of the matching result. We demonstrate our approach with the same hand motion as in the previous method and compare the performance of these two methods. We show that the MLFM method is faster and more robust to hand occlusions than the PMF method. Finally, we present a more powerful approach based on the Back-Constrained Gaussian Process Latent Variable Model (BC-GPLVM) that import prior information of categorized hand motion such as typing. BC-GPLVM provides a low-dimensional embedding of hand motion data, with a density function that gives higher probability to gestures and motions close to normal typing motion. This model is learned from a small set of clusters of vast captured motion data. The prior model is then combined with the previous multi-level models and features to generate a uniform cost function for optimization. We show the tracking result to demonstrate that this method can be used for human-computer interfaces requiring precise 3D hand tracking.

EPFL2008

Analysis of Multiview Omnidirectional Images in a Spherical Framework

Zafer Arican

With the increasing demand of information for more immersive applications such as Google Street view or 3D movies, the efficient analysis of visual data from cameras has gained more importance. This visual information permits to extract some crucial information in scenes such as similarity for recognition, 3D scene information, textures and patterns of objects. Multi-camera systems provide detailed visual information about a scene with images from different positions and viewing angles. The conventional perspective cameras that are commonly used in these systems however, have limited field of view. Therefore, they require either the deployment of many of these cameras or the capture of many images from different points to extract sufficient details about a scene. It increases the amount of data to be processed and the maintenance costs for such systems. Omnidirectional vision systems overcome this problem due to their 360-degree field of view and have found wide application areas in robotics, surveillance. These systems often use special refractive elements such as fisheye lenses or mirror-lense systems. The resulting images however inherit from the specific geometry of these units. Therefore, the analysis of these images with methods designed for perspective cameras results in degraded performance in omnivision systems. In this thesis, we focus on the analysis of multi-view omnidirectional images for efficient scene information extraction. We propose a novel spherical framework for omnidirectional image processing by exploiting the property that most omnidirectional images can be uniquely mapped on the sphere. We propose solutions for three common multiview image processing problems, namely feature detection, dense depth estimation and super-resolution for omnidirectional images. We first address the feature extraction problem in omnidirectional images. We develop a scale-invariant feature detection method which carefully handles the geometry of the images by performing the scale-space analysis directly on their native manifolds such as the parabola or the sphere. We then propose a new descriptor and a matching criteria that take into account the geometry and also eliminate the need for orientation computation. We also demonstrate that the proposed method can be used to match features in images captured by different types of sensors such as perspective, omnidirectional or spherical cameras. We then propose a dense depth estimation method to extract the 3D scene information from multiple omnidirectional images. We propose a graph-cut method adapted to the geometry of those images to minimize an energy function formulated for the dense depth estimation problem. We also propose a parallel graph-cut method that gives a significant speed improvement without a big penalty in accuracy. We show that the proposed method can be applied to multi-camera depth estimation and depth-based arbitrary view synthesis. Finally, we consider the multi-view omnidirectional images with the transformation of pure rotation. We address the view synthesis problem for these images in the framework of super-resolution. Considering also the inaccuracies in the rotation parameters, we solve an optimization problem that jointly estimates the rotation errors and reconstructs a high resolution omnidirectional image. We then extend the minimization problem with a regularization term for improved reconstruction quality with a reduced number of images. Results with both synthetic and real omnidirectional images suggest that the proposed method is a viable solution for super-resolution with omnidirectional images. Overall, this dissertation addresses three important issues of multiview omnidirectional image analysis and processing in a novel spherical framework. Our feature detection method can be used for the calibration of omnidirectional images as well as the feature matching in mobile and hybrid camera networks. Furthermore, our dense depth estimation method can impact the quality of 3D scene reconstruction and provide efficient solutions for view synthesis and multiview omnidirectional image coding. Finally, our super-resolution algorithm for omnidirectional images can promote the development of efficient acquisition systems for high resolution omnidirectional images

EPFL2010

Comparative analysis of transmittance measurement geometries and apparatus

Marjan Shahpaski, Sabine Süsstrunk

The accurate measurement of reflectance and transmittance properties of materials is essential in the printing and display industries in order to ensure precise color reproduction. In comparison with reflectance measurement, where the impact of different geometries (0°/45°, d/8°) has been thoroughly investigated, there are few published articles related to transmittance measurement. In this work, we explore different measurement geometries for total transmittance, and show that the transmittance measurements are highly affected by the geometry used, since certain geometries can introduce a measurement bias. We present a flexible custom setup that can simulate these geometries, which we evaluate both qualitatively and quantitatively over a set of samples with varied optical properties. We also compare our measurements against those of widely used commercial solutions, and show that significant differences exist over our test set. However, when the bias is correctly compensated, very low differences are observed. These findings therefore stress the importance of including the measurement geometry when reporting total transmittance.