Face authentication based on local features and generative models

The principal objective of this thesis is to investigate approaches toward a robust automatic face authentication (AFA) system in weakly constrained environments. In this context, we develop new algorithms based on local features and generative models. In addition, particular attention is given to face localization which is a necessary step of a fully automatic system. In an authentication scenario, a person claims an identity and, using one or several face images to support this claim, the system classifies the person as either a true claimant (called client) or as an impostor. Unlike face identification, the face authentication task aims to assign a given face image into one of two classes. This task is particularly difficult since any person can be encountered; ie. the impostors have usually not been seen before. One of the other major challenges of AFA is the lack of reference images. Indeed, it is not realistic to have a huge amount of images for each identity. Usually, only one or a few images are available and they can not cover all the possible variabilities due to different expression, lighting, background, head pose, hair cut, etc. Generative models such as Gaussian mixture models (GMMs), one-dimensional hidden Markov models (1D-HMMs) and pseudo two-dimensional hidden Markov models (P2D-HMMs) have proved to be efficient for face identification. In this thesis, we propose to train generative models using maximum a posteriori (MAP) training instead of the traditionally used maximum likelihood (ML) criterion. We experimentally demonstrate the superiority of this approach over other training schemes. The main motivation for the use of MAP training is the ability of this algorithm to estimate robust model parameters when there is only a few training images available. Using P2D-HMM trained with MAP, we obtain better performance than state-of-the-art face authentication approaches. In a second part of this thesis, we proposed some improvements of the baseline systems in order to increase performances with minimal effects in computation time. The first proposition is to extend the feature vectors for the GMM approach in order to embed positional information. This new system improves slightly the performances comparing to the baseline GMM approach. The second proposed approach is an alternative 1D-HMM topology which allows the use of observation vectors representing image blocks instead a whole line for standard 1D-HMM implementation. The experiments demonstrate that this model is significantly more robust than the standard 1D-HMM. Due to is low complexity, it is also eight times faster than a P2D-HMM with the cost of a lower accuracy. Finally, in the last part of the thesis, we propose a new methodology to evaluate face localization algorithms in the context of face authentication. We first show the influence of localization errors on face authentication systems and then empirically demonstrate the problems of current localization performance m