Recognizing Human Visual Focus of Attention from Head Pose in Meetings

We address the problem of recognizing the visual focus of attention (VFOA) of meeting participants based on their head pose. To this end, the head pose observations are modeled using a Gaussian Mixture Model (GMM) or a Hidden Markov Model (HMM) whose hidden states corresponds to the VFOA. The novelties of this work are threefold. First, contrary to previous studies on the topic, in our set-up, the potential VFOA of a person is not restricted to other participants only. It includes environmental targets as well (a table and a projection screen), which increases the complexity of the task, with more VFOA targets spread in the pan as well as tilt gaze space. Second, we propose a geometric model to set the GMM or HMM parameters by exploiting results from cognitive science on saccadic eye motion, which allows the prediction of the head pose given a gaze target. Third, an unsupervised parameter adaptation step not using any labeled data is proposed which accounts for the specific gazing behaviour of each participant.

Bayesian networks as generative models for face recognition

Guillaume Heusch

In this thesis, we address the problem of face modelling by using dedicated statistical generative models, with an application to the face authentication task. Face authentication consists in either accepting or rejecting a user's claim supported by its face image. Classical generative models, such as Gaussian Mixture Models (GMM), Hidden Markov Models (HMM) and their variants have been proved to be successful to tackle this problem. However, these models are not appropriate to the structure of the observed data. In particular, these models implicitly assume independence between features extracted from the image, which is obviously not true in the case of a face. We thus propose new generative models, based on Bayesian Networks, and especially tailored to deal with the object we have to handle: the face. Actually, we would like to exploit as much as a priori knowledge as possible. For this purpose, Bayesian Networks provide an intuitive framework: they allow to encode causal relationships between different kind of random variables, thus enabling to express correlations between different source of information. As a first step, we thus propose a model acting on local observations extracted around salient facial features, which is designed to capture relationships among these pieces of information. The proposed model is shown to be competitive with state-of-the-art approaches based on generative models when applied to the authentication task. At the same level of performance, it is also less complex and thus less time consuming than previous approaches. Besides, and as opposed to classical models, meaningful information could be retrieved from the proposed model. We then extend this model using other sources of information as complementary clues to local features extracted from grayscale face images. Indeed, cognitive studies in face recognition showed that human beings are using various information such as shape, low-resolution representation of the face and also skin color to recognize an individual. We thus proposed new models taking these information into account, and apply them to our authentication problem.

EPFL2010

Bayesian networks as generative models for face recognition

Guillaume Heusch

EPFL2010