Publications related to Stochastic Three-Composite Convex Minimization with a Linear Operator

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Stochastic homogenization of degenerate integral functionals with linear growth

Matthias Ruf

We study the limit behaviour of sequences of non-convex, vectorial, random integral functionals, defined on W1,1, whose integrands are ergodic and satisfy degenerate linear growth conditions. The latter involve suitable random, scale-dependent weight-funct ...

2023

Stochastic distributed learning with gradient quantization and double-variance reduction

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

TAYLOR & FRANCIS LTD2022

Proximal Point Imitation Learning

Volkan Cevher, Luca Viano, Igor Krawczuk, Angeliki Kamoutsi

This work develops new algorithms with rigorous efficiency guarantees for infinite horizon imitation learning (IL) with linear function approximation without restrictive coherence assumptions. We begin with the minimax formulation of the problem and then o ...

2022

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

SIAM PUBLICATIONS2022

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Mireille El Gheche, Giovanni Chierchia

Proximal splitting methods are standard tools for nonsmooth optimization. While primal-dual methods have become very popular in the last decade for their flexibility, primal methods may still be preferred for two reasons: acceleration schemes are more effe ...

IEEE2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

2021

Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation

Vincent Gramoli

There is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL), in which the model training is distributed over mobile user equipment (UEs), exploiting UEs' local computation and training data. Despite its advan ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Fast Projection Onto Convex Smooth Constraints

Maryam Kamgarpour, Andreas Krause, Ilnura Usmanova

The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where ...

PMLR2021

Stochastic Three-Composite Convex Minimization with a Linear Operator

Graph Chatbot

Chat with Graph Search

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Stochastic homogenization of degenerate integral functionals with linear growth

Stochastic distributed learning with gradient quantization and double-variance reduction

Proximal Point Imitation Learning

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

A first-order primal-dual method with adaptivity to local smoothness

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation

Fast Projection Onto Convex Smooth Constraints

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Proximal Point Imitation Learning

Stochastic distributed learning with gradient quantization and double-variance reduction

Stochastic homogenization of degenerate integral functionals with linear growth

A first-order primal-dual method with adaptivity to local smoothness

Yapa: Accelerated Proximal Algorithm For Convex Composite Problems

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Fast Projection Onto Convex Smooth Constraints

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation