Publications related to A Catalyst Framework for Minimax Optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Baiwei Guo

Control systems operating in real-world environments often face disturbances arising from measurement noise and model mismatch. These factors can significantly impact the perfor- mance and safety of the system. In this thesis, we aim to leverage data to de ...

EPFL2023

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Nicolas Boumal

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. This turns out to be unexpectedly delicate. We trace the difficulty back to a geometric obstacle: On a nonsmooth se ...

SPRINGER HEIDELBERG2022

Stochastic distributed learning with gradient quantization and double-variance reduction

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

TAYLOR & FRANCIS LTD2022

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Convex optimization in sums of Banach spaces

Michaël Unser, Shayan Aziznejad

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of compo ...

2021

A Catalyst Framework for Minimax Optimization

Negar Kiyavash

We introduce a generic two-loop scheme for smooth minimax optimization with strongly-convex-concave objectives. Our approach applies the accelerated proximal point framework (or Catalyst) to the associated dual problem and takes full advantage of existing ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2020

A Catalyst Framework for Minimax Optimization

Graph Chatbot

Chat with Graph Search

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Stochastic distributed learning with gradient quantization and double-variance reduction

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

A first-order primal-dual method with adaptivity to local smoothness

Convex optimization in sums of Banach spaces

A Catalyst Framework for Minimax Optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

A first-order primal-dual method with adaptivity to local smoothness

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Convex optimization in sums of Banach spaces

Stochastic distributed learning with gradient quantization and double-variance reduction

A Catalyst Framework for Minimax Optimization