Learning to Learn, from Transfer Learning to Domain Adaptation: A Unifying Perspective

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Transfer Learning by Sharing Support Vectors

Vitaly Henry Ablavsky, Carlos Joaquin Becker, Pascal Fua

Machine Learning techniques play an increasingly vital role in the analysis of Biomedical imagery, as in all other areas of Computer Vision. However, in this specific context, they suffer from the fact that experimental conditions and protocols change often and that acquiring sufficient amounts of new training data after each image acquisition is impractical. In this paper, we propose an effective method to train a non-linear SVM using a very small amount of new data by leveraging data obtained under different conditions. Unlike earlier approaches, ours takes full advantage of the kernelized SVM formulation, does not depend on a loss function that is sensitive to outliers, and yields a quadratic optimization problem. We demonstrate its effectiveness for the purpose of classifying pixels in electron microscope image stacks and delineating linear structures in optical microscopy and retinal scans. Our method outperforms two state-of-the-art transfer-learning approaches in terms of accuracy and computational complexity.

2012

Transfer Learning of Visual Concepts across Robots: a Discriminative Approach

Barbara Caputo, Tatiana Tommasi

While there is a general consensus that autonomous robots should be able to learn continuously over time, the learning process is traditionally envisioned for each specific robot situated in a given environment. This does not consider the fact that robots performing similar tasks in similar settings would probably learn similar concepts. They would therefore benefit by sharing their prior experience with each other. In this paper we present a transfer learning algorithm that enables robots located in different places to take advantage of each other’s experience, boosting the learning process. We specifically assume to have robots equipped with a camera. We do not make any assumption on the type of camera, nor on where it is positioned. We also assume that the robots use the same feature descriptors and learning algorithms. Under these assumptions, we show that one robot can hugely benefit from what has been learned by peer robots performing similar tasks. The advantage concretely means a consistent boost in performance, especially when training data is scarce. Our algorithm is based on Least Square Support Vector Machine, and allows to determine automatically from where to transfer and how much to transfer: this makes it possible to take advantage of the prior when it is useful, while minimizing the risk of negative transfer when the priors are not informative. Thorough experiments on four different publicly available databases show the power of our approach.

Idiap2012

Theory and Algorithms for Hypothesis Transfer Learning

Ilja Kuzborskij

The design and analysis of machine learning algorithms typically considers the problem of learning on a single task, and the nature of learning in such scenario is well explored. On the other hand, very often tasks faced by machine learning systems arrive sequentially, and therefore it is reasonable to ask whether a better approach can be taken than retraining such systems from scratch given newly available data. Indeed, by drawing analogy from human learning, a novel skill could be acquired more easily whenever the learner shares a relevant past experience. In response to this observation, the machine learning community has drawn its attention towards a form of learning known as transfer learning - learning a novel task by leveraging upon auxiliary information extracted from previous tasks. Tangible progress has been made in both theory and practice of transfer learning; however, many questions are still to be addressed. In this thesis we will focus on an efficient type of transfer learning, known as the Hypothesis Transfer Learning (HTL), where auxiliary information is retained in a form of previously induced hypotheses. This is in contrast to the large body of work where one transfers from the data associated with previously encountered tasks. In particular, we theoretically investigate conditions when HTL guarantees improved generalization on a novel task subject to the relevant auxiliary (source) hypotheses. We investigate HTL theoretically by considering three scenarios: HTL through regularized least squares with biased regularization, through convex empirical risk minimization, and through stochastic optimization, which also touches the theory of non-convex transfer learning problems. In addition, we demonstrate the benefits of HTL empirically, by proposing two algorithms tailored for real-life situations with application to visual learning problems - learning a new class in a multi-class classification setting by transferring from known classes, and an efficient greedy HTL algorithm for learning with large number of source hypotheses. From theoretical point of view this thesis consistently identifies the key quantitative characteristics of relatedness between novel and previous tasks, and explicitates them in generalization bounds. These findings corroborate many previous works in the transfer learning literature and provide a theoretical basis for design and analysis of new HTL algorithms.

EPFL2018

Theory and Algorithms for Hypothesis Transfer Learning

Ilja Kuzborskij

EPFL2018