Scale-invariant feature transform

The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local features in images, invented by David Lowe in 1999. Applications include object recognition, robotic mapping and navigation, , 3D modeling, gesture recognition, video tracking, individual identification of wildlife and match moving. SIFT keypoints of objects are first extracted from a set of reference images and stored in a database. An object is recognized in a new image by individually comparing each feature from the new image to this database and finding candidate matching features based on Euclidean distance of their feature vectors. From the full set of matches, subsets of keypoints that agree on the object and its location, scale, and orientation in the new image are identified to filter out good matches. The determination of consistent clusters is performed rapidly by using an efficient hash table implementation of the ge

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

The ever increasing number of digital images in both public and private collections urges on the need for generic image content analysis systems. These systems need to be capable to capture the content of images from both scenes and objects, in a compact way that allows for fast search and comparison. Modeling images based on local invariant features computed at interest point locations has proven in recent years to achieve such capabilities and to provide a robust and versatile way to perform wide-baseline matching and search for both scene and object images. In this thesis we explore the use of local descriptors for image representation in the tasks of scene and object classification, ranking, and segmentation. More specifically, we investigate the combined use of text modeling methods and local invariant features. Firstly, our work attempts to elucidate whether a text like bag-of-visterms representation (histogram of quantized local visual features) is suitable for scene and object classification, and whether some analogies between discrete scene representations and text documents exist. We further explore the bag-of-visterms approach in a fusion framework, combining texture and color information for natural scene classification. Secondly, we investigate whether unsupervised, latent space models can be used as feature extractors for the classification task and to discover patterns of visual co-occurrence. In this direction, we show that Probabilistic Latent Semantic Analysis (PLSA) generates a compact scene representation, discriminative for accurate classification, and more robust than the bagof-visterms representation when less labeled training data is available. Furthermore, we show through aspect-based image ranking experiments, the ability of PLSA to automatically extract visually meaningful scene patterns, making such representation useful for browsing image collections. Finally, we further explore the use of the latent aspect modeling in an image segmentation task. By extending the representation resulting from the latent aspect modeling, we are able to introduce contextual information for image segmentation that goes beyond the traditional regional contextual modeling found for instance in Markov Random Field approaches.

École Polytechnique Fédérale de Lausanne2007

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

EPFL2007

On Local Features for Face Verification

{W}e compare four local feature extraction techniques for the task of face verification, namely (ordered in terms of complexity): raw pixels, raw pixels with mean removal, 2D Discrete Cosine Transform (DCT) and local Principal Component Analysis (PCA). The comparison is performed in terms of discrimination ability and robustness to illumination changes. We also evaluate the effectiveness of several approaches to modifying standard feature extraction methods in order to increase performance and robustness to illumination changes. Results on the XM2VTS database suggest that when using a Gaussian Mixture Model (GMM) based classifier, the raw pixel technique provides poor discrimination and is easily affected by illumination changes; the mean removed raw pixel technique provides performance that is fairly close to 2D DCT and local PCA, but is considerably affected by illumination changes. The performance of 2D DCT and local PCA techniques is quite similar, suggesting that the 2D DCT technique is to be preferred over the local PCA technique, due to the lower complexity of the 2D DCT. Both 2D DCT and local PCA techniques are considerably more robust to illumination changes compared to the raw pixel techniques. Modifying the 2D DCT and local PCA techniques by removing the first coefficient, which is deemed to be the most affected by illumination changes, clearly enhances robustness; removing more than the first coefficient causes a noticeable reduction in performance on clean images and provides no further gains in robustness. Compared to just throwing out the first coefficient, the use of deltas can achieve a small increase in performance and robustness. Lastly, we suggest that it is more appropriate to use analysis blocks of size 8x8 (as opposed to 16x16) with 2D DCT decomposition; out of the 64 resulting coefficients, the second through to 21-st (resulting in 20 dimensional feature vectors) are the most robust to illumination changes while providing good discriminatory information.

IDIAP2004

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

École Polytechnique Fédérale de Lausanne2007

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

EPFL2007

On Local Features for Face Verification

IDIAP2004