Automatic face analysis in static and dynamic environments

Faces play a vital role in people's daily lives, as they are able to convey many different information and above all their behaviour and emotions. Therefore, for decades the face has been investigated by researchers in several fields such as computer science, psychology and social science to the extent that it became recently a fundamentally multidisciplinary matter. In order to study the perception and interpretation of this information in human-human, as well as human-computer interaction, the analysis of the face is therefore essential. However, considering that computers are unable to communicate affect, one of the today's major research challenges is to "humanize" them by recognizing and reproducing emotional states. For this reason in the last decade research in computer vision has increasingly focused on facial expression analysis. In this thesis, we concentrate on face analysis by examining the content, firstly, of images and, secondly, of video. In the first case the goal is to define models that are able to take into account the heterogeneity of human perception of static facial expressions. Data collected through a web-based survey reveals that the perception of these expressions by a human observer is a subjective process, which is interpreted here as a choice process where the decision-maker is considered to be rational. The discrete choice analysis technique well fits the evaluation process of the human observer and it is used here as the modelling mathematical framework. In the second case we extend the face analysis to a dynamic environment by developing a system capable of interpreting the facial responses of infants during their meals. The architecture integrates image-based and behavioural information in order to provide food scientists with an automatic framework capable of analysing infant's reaction at each offered spoon of the meal. The input of the system is then a video of an infant's meal and the output is a sequence of automatically detected spoon intervals with an associated liking rating which quantifies the infant's appreciation to the presented food.