Visage | EPFL Graph Search

thumb|right|200px|Visage de jeune fille (peinture d'étude de William Bouguereau - ). Le visage est la zone externe de la partie antérieure de la tête de l'être humain, appelée aussi face ou figure. Il se structure autour de zones osseuses abritant plusieurs organes des sens ; il comprend notamment la peau, le menton, la bouche, les lèvres, le philtrum, les dents, le nez, les joues, les yeux, les sourcils, le front, les cheveux et les oreilles. Chaque visage est unique, ce qui en fait un élément essentiel de l'identité et une base pour l'identification des personnes (photographie d'identité, anthropométrie, reconnaissance faciale...). C'est un vecteur essentiel pour les interactions sociales et l'empathie, et pour la communication non verbale, notamment via à l'expression des émotions permise par la très grande mobilité et complexité des muscles orofaciaux sous contrôle du nerf facial. Les humains comme d'autres primates ont développé une habileté particulière dans la perception des v

Multimodal person recognition in audio-visual streams

Do Hoang Nam Le

Multimedia databases are growing rapidly in size in the digital age. To increase the value of these data and to enhance the user experience, there is a need to make these videos searchable through automatic indexing. Because people appearing and talking in the videos are often of high interest for end users, indices that represent the location and identity of people in the archive are indispensable for video search and browsing tools. On the other hand, multimedia videos contain resourceful data of people in both visual and auditory domains. This offers a potential for multimodal learning in the task of human identification. Hence, the main theme of this thesis is on algorithms to create indexes and exploit the audio-visual correspondence in large multimedia corpuses based on person identities. First, this thesis deals with algorithms to create indexes through person discovery in videos. It involves several components: face and speaker diarization, face-voice association, and person naming. To obtain face clusters, we propose a novel face tracking approach that leverages face detectors with a tracking-by-detection framework relying on long term time-interval sensitive association costs. We use also shot context to further accelerate and improve face clustering. Face clusters are then associated to speaker clusters using dubbing and talking detection, in which a multimodal framework is introduced to represent the temporal relationship between the auditory and visual streams. We also improve speaker embeddings for recognition and clustering by using a regularizer called intra-class loss. In the second half, the thesis focuses on multimodal learning with face-voice data. Here, we aim to answer two research questions. First, can one improve a voice embedding using knowledge transferred from a face representation? We investigate several transfer learning approaches to constrain the target voice embedding space to share latent attributes with the source face embedding space. The crossmodal constrains act as regularizers helping voice models, especially in the low-data setting. The second question is can face clusters be used as training labels to learn a speaker embedding? To answer this, we explore the tolerance of embedding losses under label uncertainty. From the risk minimization perspective, we obtain the analytical results that provide the heuristics in strategies to improve the tolerance against label noise. We apply the findings into our task of learning speaker embeddings using face clusters as labels. While the experimental results agree with the analytical heuristics, there is still a large gap in performance between the supervised and the weakly supervised models, which requires further investigation in the future.

EPFL2019

Face Detection and Verification using Local Binary Patterns

Yann Rodriguez

This thesis proposes a robust Automatic Face Verification (AFV) system using Local Binary Patterns (LBP). AFV is mainly composed of two modules: Face Detection (FD) and Face Verification (FV). The purpose of FD is to determine whether there are any face in an image, while FV involves confirming or denying the identity claimed by a person. The contributions of this thesis are the following: 1) a real-time multiview FD system which is robust to illumination and partial occlusion, 2) a FV system based on the adaptation of LBP features, 3) an extensive study of the performance evaluation of FD algorithms and in particular the effect of FD errors on FV performance. The first part of the thesis addresses the problem of frontal FD. We introduce the system of Viola and Jones which is the first real-time frontal face detector. One of its limitations is the sensitivity to local lighting variations and partial occlusion of the face. In order to cope with these limitations, we propose to use LBP features. Special emphasis is given to the scanning process and to the merging of overlapped detections, because both have a significant impact on the performance. We then extend our frontal FD module to multiview FD. In the second part, we present a novel generative approach for FV, based on an LBP description of the face. The main advantages compared to previous approaches are a very fast and simple training procedure and robustness to bad lighting conditions. In the third part, we address the problem of estimating the quality of FD. We first show the influence of FD errors on the FV task and then empirically demonstrate the limitations of current detection measures when applied to this task. In order to properly evaluate the performance of a face detection module, we propose to embed the FV into the performance measuring process. We show empirically that the proposed methodology better matches the final FV performance.

IDIAP2006

Face Detection and Verification using Local Binary Patterns

Yann Rodriguez

École Polytechnique Fédérale de Lausanne2006

Multimodal person recognition in audio-visual streams

Do Hoang Nam Le

EPFL2019

Face Detection and Verification using Local Binary Patterns

Yann Rodriguez

IDIAP2006

Face Detection and Verification using Local Binary Patterns

Yann Rodriguez

École Polytechnique Fédérale de Lausanne2006