Concept

Stochastic gradient descent

Related publications (431)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Stochastic Spectral Descent for Discrete Graphical Models

Volkan Cevher, Ya-Ping Hsieh, Edo Collins

Interest in deep probabilistic graphical models has increased in recent years, due to their state-of-the-art perfor- mance on many machine learning applications. Such models are typically trained with the stochastic gradient method, which can take a signif ...

Ieee-Inst Electrical Electronics Engineers Inc2016

On the influence of momentum acceleration on online learning

Ali H. Sayed, Bicheng Ying, Kun Yuan

This paper examines the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime. The results establish that momentum methods are equivalent to the standard stochastic g ...

IEEE2016

Stochastic Three-Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Cong Bang Vu

We propose a stochastic optimization method for the minimization of the sum of three convex functions, one of which has Lipschitz continuous gradient as well as restricted strong convexity. Our approach is most suitable in the setting where it is computati ...

2016

Principled Parallel Mean-Field Inference for Discrete Random Fields

Pascal Fua, François Fleuret, Pierre Bruno Baqué, Timur Bagautdinov

Mean-field variational inference is one of the most popular approaches to inference in discrete random fields. Standard mean-field optimization is based on coordinate descent and in many situations can be impractical. Thus, in practice, various parallel te ...

2016

Adaptive data augmentation for image classification

Pascal Frossard, Alhussein Fawzi

Data augmentation is the process of generating samples by transforming training data, with the target of improving the accuracy and robustness of classifiers. In this paper, we propose a new automatic and adaptive algorithm for choosing the transformations ...

IEEE2016

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same f ...

EPFL2016

Accelerated ADMM based on Accelerated Douglas-Rachford Splitting

Colin Neil Jones, Ivan Pejcic

Alternating direction method of multipliers (ADMM) is a form of augmented Lagrangian optimisation algorithm that found its place in many new applications in recent years. This paper explores a possibility for an upgrade of the ADMM by extrapolation-based a ...

Ieee2016

Robust Online Time Series Prediction with Recurrent Neural Networks

Karl Aberer, Tian Guo

Time series forecasting for streaming data plays an important role in many real applications, ranging from IoT systems, cyber-networks, to industrial systems and healthcare. However the real data is often complicated with anomalies and change points, which ...

Ieee2016

Fast rates by transferring from auxiliary hypotheses

Ilja Kuzborskij

In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated ...

2016

Stochastic learning and control in multiple coordinate systems

Sylvain Calinon

A probabilistic interpretation of model predictive control is presented, enabling extensions to multiple coordinate systems. The resulting controller follows a minimal intervention principle, by learning and retrieving movements through the coordination of ...

2016