On-line Independent Support Vector Machines for Cognitive Systems

Learning from experience and adapting to changing stimuli are fundamental capabilities for artificial cognitive systems. This calls for on-line learning methods able to achieve high accuracy while at the same time using limited computer power. Research on autonomous agents has been actively investigating these issues, mostly using probabilistic frameworks and within the context of navigation and learning by imitation. Still, recent results on robot localization have clearly pointed out the potential of discriminative classifiers for cognitive systems. In this paper we follow this approach and propose an on-line version of the Support Vector Machine (SVM) algorithm. Our method, that we call On-line Independent SVM, builds a solution on-line, achieving an excellent accuracy vs.~compactness trade-off. In particular the size of the obtained solution is always bounded, implying a bounded testing time. At the same time, the algorithm converges to the optimal solution at each incremental step, as opposed to similar approaches where optimality is achieved in the limit of infinite number of training data. These statements are supported by experiments on standard benchmark databases as well as on two real-world applications, namely $(a)$ place recognition by a mobile robot in an indoor environment, and $(b)$ human grasping posture classification.

Learning embeddings: efficient algorithms and applications

Cijo Jose

Learning to embed data into a space where similar points are together and dissimilar points are far apart is a challenging machine learning problem. In this dissertation we study two learning scenarios that arise in the context of learning embeddings and one scenario in efficiently estimating an empirical expectation. We present novel algorithmic solutions and demonstrate their applications on a wide range of data-sets. The first scenario deals with learning from small data with large number of classes. This setting is common in computer vision problems such as person re-identification and face verification. To address this problem we present a new algorithm called Weighted Approximate Rank Component Analysis (WARCA), which is scalable, robust, non-linear and is independent of the number of classes. We empirically demonstrate the performance of our algorithm on 9 standard person re-identification data-sets where we obtain state of the art performance in terms of accuracy as well as computational speed. The second scenario we consider is learning embeddings from sequences. When it comes to learning from sequences, recurrent neural networks have proved to be an effective algorithm. However there are many problems with existing recurrent neural networks which makes them data hungry (high sample complexity) and difficult to train. We present a new recurrent neural network called Kronecker Recurrent Units (KRU), which addresses the issues of existing recurrent neural networks through Kronecker matrices. We show its performance on 7 applications, ranging from problems in computer vision, language modeling, music modeling and speech recognition. Most of the machine learning algorithms are formulated as minimizing an empirical expectation over a finite collection of samples. In this thesis we also investigate the problem of efficiently estimating a weighted average over large data-sets. We present a new data-structure called Importance Sampling Tree (IST), which permits fast estimation of weighted average without looking at all the samples. We show successfully the evaluation of our data-structure in the training of neural networks in order to efficiently find informative samples.

EPFL2018

Learning embeddings: efficient algorithms and applications

Cijo Jose

École Polytechnique Fédérale de Lausanne2018

Transfer Learning of Visual Concepts across Robots: a Discriminative Approach

Barbara Caputo, Tatiana Tommasi

While there is a general consensus that autonomous robots should be able to learn continuously over time, the learning process is traditionally envisioned for each specific robot situated in a given environment. This does not consider the fact that robots performing similar tasks in similar settings would probably learn similar concepts. They would therefore benefit by sharing their prior experience with each other. In this paper we present a transfer learning algorithm that enables robots located in different places to take advantage of each other’s experience, boosting the learning process. We specifically assume to have robots equipped with a camera. We do not make any assumption on the type of camera, nor on where it is positioned. We also assume that the robots use the same feature descriptors and learning algorithms. Under these assumptions, we show that one robot can hugely benefit from what has been learned by peer robots performing similar tasks. The advantage concretely means a consistent boost in performance, especially when training data is scarce. Our algorithm is based on Least Square Support Vector Machine, and allows to determine automatically from where to transfer and how much to transfer: this makes it possible to take advantage of the prior when it is useful, while minimizing the risk of negative transfer when the priors are not informative. Thorough experiments on four different publicly available databases show the power of our approach.

Idiap2012

Learning embeddings: efficient algorithms and applications

Cijo Jose

École Polytechnique Fédérale de Lausanne2018

Learning embeddings: efficient algorithms and applications

Cijo Jose

EPFL2018