Visual descriptor | EPFL Graph Search

In computer vision, visual descriptors or image descriptors are descriptions of the visual features of the contents in s, videos, or algorithms or applications that produce such descriptions. They describe elementary characteristics such as the shape, the color, the texture or the motion, among others. Introduction As a result of the new communication technologies and the massive use of Internet in our society, the amount of audio-visual information available in digital format is increasing considerably. Therefore, it has been necessary to design some systems that allow us to describe the content of several types of multimedia information in order to search and classify them. The audio-visual descriptors are in charge of the contents description. These descriptors have a good knowledge of the objects and events found in a video, image or audio and they allow the quick and efficient searches of the audio-visual content. This system can be compared to the search engines for

G2-VER: Geometry Guided Model Ensemble for Video-based Facial Expression Recognition

Tanguy Albrici, Hazim Kemal Ekenel, Jean-Philippe Thiran, Guillaume Marc Georges Vray

This paper addresses the problem of automatic facial expression recognition in videos, where the goal is to predict discrete emotion labels best describing the emotions expressed in short video clips. Building on a pre-trained convolutional neural network ( CNN) model dedicated to analyzing the video frames and LSTM network designed to process the trajectories of the facial landmarks, this paper investigates several novel directions. First of all, improved face descriptors based on 2D CNNs and facial landmarks are proposed. Second, the paper investigates fusion methods of the features temporally, including a novel hierarchical recurrent neural network combining facial landmark trajectories over time. In addition, we propose a modification to state-of-the-art expression recognition architectures to adapt them to video processing in a simple way. In both ensemble approaches, the temporal information is integrated. Comparative experiments on publicly available video-based facial expression recognition datasets verified that the proposed framework outperforms state-of-the-art methods. Moreover, we introduce a near-infrared video dataset containing facial expressions from subjects driving their cars, which are recorded in real world conditions.

IEEE2019

Variational methods for texture segmentation

Nawal Houhou

In the last decades, image production has grown significantly. From digital photographs to the medical scans, including satellite images and video films, more and more data need to be processed. Consequently the number of applications based on digital images has increased, either for medicine, research for country planning or for entertainment business such as animation or video games. All these areas, although very different one to another, need the same image processing techniques. Among all these techniques, segmentation is probably one of the most studied because of its important role. Segmentation is the process of extracting meaningful objects from an image. This task, although easily achieved by the human visual system, is actually complex and still a true challenge for the image processing community despite several decades of research. The thesis work presented in this manuscript proposes solutions to the image segmentation problem in a well established mathematical framework, i.e. variational models. The image is defined in a continuous space and the segmentation problem is expressed through a functional or energy optimization. Depending on the object to be segmented, this energy definition can be difficult; in particular for objects with ambiguous borders or objects with textures. For the latter, the difficulty lies already in the definition of the term texture. The human eye can easily recognize a texture, but it is quite difficult to find words to define it, even more in mathematical terms. There is a deliberate vagueness in the definition of texture which explains the difficulty to conceptualize a model able to describe it. Often these textures can neither be described by homogeneous regions nor by sharp contours. This is why we are first interested in the extraction of texture features, that is to say, finding one representation that can discriminate a textured region from another. The first contribution of this thesis is the construction of a texture descriptor from the representation of the image similar to a surface in a volume. This descriptor belongs to the framework of non-supervised segmentation, since it will not require any user interaction. The second contribution is a solution for the segmentation problem based on active contour models and information theory tools. Third contribution is a semi-supervised segmentation model, i.e. where constraints provided by the user will be integrated in the segmentation framework. This processus is actually derived from the graph of image patches. This graph gives the connectivity measure between the different points of the image. The segmentation will be expressed by a graph partition and a variational model. This manuscript proposes to tackle the segmentation problem for textured images.

EPFL2009

Variational methods for texture segmentation

Nawal Houhou

EPFL2009