Publications related to Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Variational methods for finding periodic orbits in the incompressible Navier Stokes equations

Tobias Schneider, Jeremy Peter Parker

Unstable periodic orbits are believed to underpin the dynamics of turbulence, but by their nature are hard to find computationally. We present a family of methods to converge such unstable periodic orbits for the incompressible Navier-Stokes equations, bas ...

CAMBRIDGE UNIV PRESS2022

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Volkan Cevher, Kimon Antonakopoulos

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem’s regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this regard, existin ...

2022

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Nicolas Henri Bernard Flammarion, Aditya Vardhan Varre, Loucas Pillaud-Vivien

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

2021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Optimization Methods for Control: From Embedded Programmable Hardware to Data-Driven Process Optimization

Harsh Ambarishkumar Shukla

The research community has been making significant progress in hardware implementation, numerical computing and algorithm development for optimization-based control. However, there are two key challenges that still have to be overcome for optimization-base ...

EPFL2021

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Graph Chatbot

Chat with Graph Search

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Variational methods for finding periodic orbits in the incompressible Navier Stokes equations

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

A first-order primal-dual method with adaptivity to local smoothness

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Optimization Methods for Control: From Embedded Programmable Hardware to Data-Driven Process Optimization

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Variational methods for finding periodic orbits in the incompressible Navier Stokes equations

Optimization Methods for Control: From Embedded Programmable Hardware to Data-Driven Process Optimization

A first-order primal-dual method with adaptivity to local smoothness

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates