Scale-invariant feature transform

[[Fichier:Matching of two images using the SIFT method.jpg|thumb|right|alt=Exemple de mise en correspondance de deux images par la méthode SIFT : des lignes vertes relient entre eux les descripteurs communs à un tableau et une photo de ce même tableau, de moindre qualité, ayant subi des transformations. |Exemple de résultat de la comparaison de deux images par la méthode SIFT (Fantasia ou Jeu de la poudre, devant la porte d’entrée de la ville de Méquinez, par Eugène Delacroix, 1832).]] La scale-invariant feature transform (SIFT), que l'on peut traduire par « transformation de caractéristiques visuelles invariante à l'échelle », est un algorithme utilisé dans le domaine de la vision par ordinateur pour détecter et identifier les éléments similaires entre différentes (éléments de paysages, objets, personnes, etc.). Il a été développé en 1999 par le chercheur David Lowe. L'étape fondamentale de la méthode proposée par Lowe consiste à calculer ce que l'on appelle les « descripteurs SIF

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

The ever increasing number of digital images in both public and private collections urges on the need for generic image content analysis systems. These systems need to be capable to capture the content of images from both scenes and objects, in a compact way that allows for fast search and comparison. Modeling images based on local invariant features computed at interest point locations has proven in recent years to achieve such capabilities and to provide a robust and versatile way to perform wide-baseline matching and search for both scene and object images. In this thesis we explore the use of local descriptors for image representation in the tasks of scene and object classification, ranking, and segmentation. More specifically, we investigate the combined use of text modeling methods and local invariant features. Firstly, our work attempts to elucidate whether a text like bag-of-visterms representation (histogram of quantized local visual features) is suitable for scene and object classification, and whether some analogies between discrete scene representations and text documents exist. We further explore the bag-of-visterms approach in a fusion framework, combining texture and color information for natural scene classification. Secondly, we investigate whether unsupervised, latent space models can be used as feature extractors for the classification task and to discover patterns of visual co-occurrence. In this direction, we show that Probabilistic Latent Semantic Analysis (PLSA) generates a compact scene representation, discriminative for accurate classification, and more robust than the bagof-visterms representation when less labeled training data is available. Furthermore, we show through aspect-based image ranking experiments, the ability of PLSA to automatically extract visually meaningful scene patterns, making such representation useful for browsing image collections. Finally, we further explore the use of the latent aspect modeling in an image segmentation task. By extending the representation resulting from the latent aspect modeling, we are able to introduce contextual information for image segmentation that goes beyond the traditional regional contextual modeling found for instance in Markov Random Field approaches.

École Polytechnique Fédérale de Lausanne2007

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

EPFL2007

On Local Features for Face Verification

{W}e compare four local feature extraction techniques for the task of face verification, namely (ordered in terms of complexity): raw pixels, raw pixels with mean removal, 2D Discrete Cosine Transform (DCT) and local Principal Component Analysis (PCA). The comparison is performed in terms of discrimination ability and robustness to illumination changes. We also evaluate the effectiveness of several approaches to modifying standard feature extraction methods in order to increase performance and robustness to illumination changes. Results on the XM2VTS database suggest that when using a Gaussian Mixture Model (GMM) based classifier, the raw pixel technique provides poor discrimination and is easily affected by illumination changes; the mean removed raw pixel technique provides performance that is fairly close to 2D DCT and local PCA, but is considerably affected by illumination changes. The performance of 2D DCT and local PCA techniques is quite similar, suggesting that the 2D DCT technique is to be preferred over the local PCA technique, due to the lower complexity of the 2D DCT. Both 2D DCT and local PCA techniques are considerably more robust to illumination changes compared to the raw pixel techniques. Modifying the 2D DCT and local PCA techniques by removing the first coefficient, which is deemed to be the most affected by illumination changes, clearly enhances robustness; removing more than the first coefficient causes a noticeable reduction in performance on clean images and provides no further gains in robustness. Compared to just throwing out the first coefficient, the use of deltas can achieve a small increase in performance and robustness. Lastly, we suggest that it is more appropriate to use analysis blocks of size 8x8 (as opposed to 16x16) with 2D DCT decomposition; out of the 64 resulting coefficients, the second through to 21-st (resulting in 20 dimensional feature vectors) are the most robust to illumination changes while providing good discriminatory information.

IDIAP2004

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

École Polytechnique Fédérale de Lausanne2007

Scene image classification and segmentation with quantized local descriptors and latent aspect modeling

Pedro Manuel Da Silva Quelhas

EPFL2007

On Local Features for Face Verification

IDIAP2004