Video tracking | EPFL Graph Search

Video tracking is the process of locating a moving object (or multiple objects) over time using a camera. It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video communication and compression, augmented reality, traffic control, medical imaging and video editing. Video tracking can be a time-consuming process due to the amount of data that is contained in video. Adding further to the complexity is the possible need to use object recognition techniques for tracking, a challenging problem in its own right. Objective The objective of video tracking is to associate target objects in consecutive video frames. The association can be especially difficult when the objects are moving fast relative to the frame rate. Another situation that increases the complexity of the problem is when the tracked object changes orientation over time. For these situations video tracking systems usually employ a motion model which describes how the im

Robust visual tracking using feature selection

Fabien Willemin

Visual tracking has become a very important component in computer vision, but achieving a robust, reliable and real time tracking remains a real challenge.In order to improve the actual state-of-the-art, we choose to study and improve one of the most performing adaptive tracker by detection. We selected Struck [27] for this quality performance and his low computational cost that makes it real time. Inspired by the great successes of binary keypoint descriptors, we choose to apply binary description to a patch. We propose to use Multi-Block Local Binary Pattern (MB-LBP), based on its great success in face detection and description. In this work we present a technique for selecting the best features for tracking. In combination with the feature selection we propose a technique to take into account contextual information in order to increase the robustness of the tracker. We propose a solution to add scale adaptation to the algorithm, and suggest to transpose this technique to add rotation adaptation. Experimentally we validate these techniques showing that we outperform the state-of-art racking algorithms. To do that we use a benchmarking tool using 51 videos and compare our algorithm to 29 algorithms.

2014

Robotic manipulation to investigate the physical principles of biological self-organization

Fazil Emre Uslu

Self-organization is the spontaneous formation of ordered patterns and networks from a population of comparatively simple elements or individuals with no prior information on neither the formation process nor the final organization. While the construction of tissues and organs is driven by the collective action of many eukaryotic cells, at a higher scale, social insects govern massive colonies via cooperative group behavior. In biological self-organization, unlike inanimate matter, interaction rules of elements are not constant but generally evolve in time and space in a history-dependent fashion. Recent technological advances in molecular biology, genetics, microscopy, and quantitative analysis have made it possible to follow the development of tissues, organs, and entire embryos at the single-cell level with high temporal resolution. Automated video tracking systems based on identification labels allow tracking of all members in insect colonies to identify individual interactions and patterns of social organization. These studies have shown that individuals must allocate themselves to actions or tasks in a dynamic manner following simple rules that incorporate local stimuli received directly from the environment and from interactions with other individuals. The majority of the available platforms provide observation only and do not allow manipulation of conditions for probing the self-organizing systems. Application of spatiotemporally resolved physical stimuli to individuals will reveal a more complete understanding of biological self-organization.In this thesis, I have developed robotic manipulation platforms along with computational models to investigate the principles of self-organization in biological systems. I have worked on two complementary experimental models at different scales to prove the generality of my approach. The first robotic manipulation platform explores the transmission and transduction of mechanical signals within a connected fiber network and associated cellular responses. I developed a biocompatible magnetic microactuator and a magnetic control system to apply physiologically relevant traction forces to cells cultured on fibrous matrices. Along with the experimental platform, I have developed a finite element-modeling framework that simulates stresses on a digital copy of the fiber network. The second robotic manipulation platform controls the temperature on the nest floor of ant colonies. The system is integrated with a real-time tracking system that is capable of following interactions among hundreds of individuals. The ant colony is stimulated to transport their brood by changing the temperature of the next floor in a spatially patterned way. We explored how ants processed thermal signals, coordinated the transport of brood, and interacted with one another.

EPFL2022

Training Algorithms for Multiple Object Tracking

Andrii Maksai

Multiple object tracking is a crucial Computer Vision Task. It aims at locating objects of interest in the image sequences, maintaining their identities, and identifying their trajectories over time. A large portion of current research focuses on tracking pedestrians, and other types of objects, that often exhibit predictable behaviours, that allow us, as humans, to track those objects. Nevertheless, most existing approaches rely solely on simple affinity or appearance cues to maintain the identities of the tracked objects, ignoring their behaviour. This presents a challenge when objects of interest are invisible or indistinguishable for a long period of time. In this thesis, we focus on enhancing the quality of multiple object trackers by learning and exploiting the long ranging models of object behaviour. Such behaviours come in different forms, be it a physical model of the ball motion, model of interaction between the ball and the players in sports or motion patterns of pedestrians or cars, that is specific to a particular scene. In the first part of the thesis, we begin with the task of tracking the ball and the players in team sports. We propose a model that tracks both types of objects simultaneously, while respecting the physical laws of ball motion when in free fall, and interaction constraints that appear when players are in the possession of the ball. We show that both the presence of the behaviour models and the simultaneous solution of both tasks aids the performance of tracking, in basketball, volleyball, and soccer. In the second part of the thesis, we focus on motion models of pedestrian and car behaviour that emerge in the outdoor scenes. Such motion models are inherently global, as they determine where people starting from one location tend to end up much later in time. Imposing such global constraints while keeping the tracking problem tractable presents a challenge, which is why many approaches rely on local affinity measures. We formulate a problem of simultaneously tracking the objects and learning their behaviour patterns. We show that our approach, when applied in conjunction with a number of state-of-the-art trackers, improves their performance, by forcing their output to follow the learned motion patterns of the scene. In the last part of the thesis, we study a new emerging class of models for multiple object tracking, that appeared recently due to availability of large scale datasets - sequence models for multiple object tracking. While such models could potentially learn arbitrarily long ranging behaviours, training them presents several challenges. We propose a training scheme and a loss function that allows to significantly improve the quality of training of such models. We demonstrate that simply using our training scheme and loss allows to learn scoring function for trajectories, which enables us to outperform state-of-the-art methods on several tracking benchmarks.