Learning How To Recognize Faces In Heterogeneous Environments

Face recognition is a mature field in biometrics in which several systems have been proposed over the last three decades. Such systems are extremely reliable under controlled recording conditions and it has been deployed in the field in critical tasks, such as in border control and in less critical ones, such as to unlock mobile phones. However, the lack of cooperation from the subject and variations on the pose, occlusion and illumination are still open problems and significantly affect error rates. Another challenge that arose recently in face recognition research is the ability of matching faces from different image domains. Use cases encompass the matching between Visual Light images (VIS) with Near infra-red images (NIR), Visual Light images (VIS) with Thermograms or Depth maps. This match can occur even in situations where no real face exists, such as matching using sketches. This task is so called Heterogeneous Face Recognition. The key difficulty in the comparison of faces in heterogeneous conditions is that images from the same subject may differ in appearance due to changes in image domain.

In this thesis we address this problem of Heterogeneous Face Recognition (HFR). Our contributions are four-fold. First, we analyze the applicability of crafted features used in face recognition in the HFR task. Second, still working with crafted features, we propose that the variability between two image domains can be suppressed with a linear shift in the Gaussian Mixture Model (GMM) mean subspace. That encompasses inter-session variability (ISV) modeling. Third, we propose that high level features of Deep Convolutional Neural Networks trained on Visual Light images are potentially domain independent and can be used to encode faces sensed in different image domains. Fourth, large-scale experiments are conducted on several HFR databases, covering various image domains showing competitive performances.

Moreover, the implementation of all the proposed techniques are integrated into a collaborative open source software library called Bob that enforces fair evaluations and encourages reproducible research.