Publication

Stochastic gradient descent with finite samples sizes

Related publications (48)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Mean-Field methods for Structured Deep-Learning in Computer Vision

Pierre Bruno Baqué

In recent years, Machine Learning based Computer Vision techniques made impressive progress. These algorithms proved particularly efficient for image classification or detection of isolated objects. From a probabilistic perspective, these methods can predi ...

EPFL2018

Not All Samples Are Created Equal: Deep Learning with Importance Sampling

François Fleuret, Angelos Katharopoulos

Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on "informative" examples ...

Idiap2018

Mirrored Langevin Dynamics

Volkan Cevher, Paul Thierry Yves Rolland, Ya-Ping Hsieh, Ali Kavis

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design. We propose a unified framework, which is inspired by the classical mirror descent, to derive ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Mirrored Langevin Dynamics

Volkan Cevher, Paul Thierry Yves Rolland, Ya-Ping Hsieh, Ali Kavis

2018

Theory and Algorithms for Hypothesis Transfer Learning

Ilja Kuzborskij

The design and analysis of machine learning algorithms typically considers the problem of learning on a single task, and the nature of learning in such scenario is well explored. On the other hand, very often tasks faced by machine learning systems arrive ...

EPFL2018

Novel Algorithms For Clustering

James Peter Newling

Clustering is a method for discovering structure in data, widely used across many scientific disciplines. The two main clustering problems this dissertation considers are K-means and K-medoids. These are NP-hard problems in the number of samples and cluste ...

EPFL2018

On the linear convergence of the stochastic gradient method with constant step-size

Volkan Cevher, Cong Bang Vu

The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size γ (SGM-CS). In this paper, we provide a necessary condition, for the linear convergence of SGM-CS, t ...

2018

On the performance of random reshuffling in stochastic learning

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

In empirical risk optimization, it has been observed that gradient descent implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data randomly and independently of each other. ...

2017

Genetic Optimization of Training Sets for Improved Machine Learning Models of Molecular Properties

Anatole von Lilienfeld, Nicholas John Browning

The training of molecular models of quantum mechanical properties based on statistical machine learning requires large data sets which exemplify the map from chemical structure to molecular property. Intelligent a priori selection of training examples is o ...

2017

Safe Adaptive Importance Sampling

Martin Jaggi, Sebastian Urban Stich, Anant Raj

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

2017