Publication

Convex Optimization using Sparsified Stochastic Gradient Descent with Memory

Publications associées (78)

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search

Stochastic Frank-Wolfe for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed fo ...

2019

Sampling can be faster than optimization

Nicolas Henri Bernard Flammarion

Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years. There is, however, limited theoretical understanding of the relati ...

PNAS2019

An accelerated communication-efficient primal-dual optimization framework for structured machine learning

Martin Jaggi, Chenxin Ma

Distributed optimization algorithms are essential for training machine learning models on very large-scale datasets. However, they often suffer from communication bottlenecks. Confronting this issue, a communication-efficient primal-dual coordinate ascent ...

2019

Diffusion Learning In Non-Convex Environments

Ali H. Sayed, Stefan Vlaski

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, recent years have seen rising interest in the behavior of gradient-descent based algorithms in non-convex environments. Most of the works on d ...

IEEE2019

Exact Diffusion for Distributed Optimization and Learning-Part II: Convergence Analysis

Ali H. Sayed, Bicheng Ying, Kun Yuan

Part I of this paper developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to the larger set of locally balanced left-sto ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Local SGD Converges Fast and Communicates Little

Sebastian Urban Stich

Mini-batch stochastic gradient descent (SGD) is state of the art in large scale distributed training. The scheme can reach a linear speedup with respect to the number of workers, but this is rarely seen in practice as the scheme often suffers from large ne ...

2019

Stochastic Learning Under Random Reshuffling With Constant Step-Sizes

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly. Recent works have pursued ju ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Distributed Inference Over Networks Under Subspace Constraints

Ali H. Sayed, Stefan Vlaski, Roula Nassif

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that enforce the objectives across the network to lie in a low-dimensiona ...

IEEE2019

Exact Diffusion for Distributed Optimization and Learning-Part I: Algorithm Development

Ali H. Sayed, Bicheng Ying, Kun Yuan

This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range an ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Almost surely constrained convex optimization

Volkan Cevher, Ahmet Alacaoglu

We propose a stochastic gradient framework for solving stochastic composite convex optimization problems with (possibly) infinite number of linear inclusion constraints that need to be satisfied almost surely. We use smoothing and homotopy techniques to ha ...

2019