Publications related to Stochastic homogenization of degenerate integral functionals with linear growth

Stochastic distributed learning with gradient quantization and double-variance reduction

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

TAYLOR & FRANCIS LTD2022

Proximal Point Imitation Learning

Volkan Cevher, Luca Viano, Igor Krawczuk, Angeliki Kamoutsi

This work develops new algorithms with rigorous efficiency guarantees for infinite horizon imitation learning (IL) with linear function approximation without restrictive coherence assumptions. We begin with the minimax formulation of the problem and then o ...

2022

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

SIAM PUBLICATIONS2022

Fluctuation estimates for the multi-cell formula in stochastic homogenization of partitions

Matthias Ruf

In this paper we derive quantitative estimates in the context of stochastic homogenization for integral functionals defined on finite partitions, where the random surface integrand is assumed to be stationary. Requiring the integrand to satisfy in addition ...

2022

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

2021

Integral Convexity And Parabolic Systems

Bernard Dacorogna

In this work we give optimal, i.e., necessary and sufficient, conditions for integrals of the calculus of variations to guarantee the existence of solutions-both weak and variational solutions-to the associated L-2-gradient flow. The initial values are mer ...

SIAM PUBLICATIONS2020

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

Ali H. Sayed, Sulaiman A S A E Alghunaim, Ming Yan

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents. This scenario arises in many machine learning and engineering ...

IEEE2020

Convex Combination of Diffusion Strategies Over Networks

Ali H. Sayed, Jie Chen

Combining diffusion strategies with complementary properties enables enhanced performance when they can be run simultaneously. In this article, we first propose two schemes for the convex combination of two diffusion strategies, namely, the power-normalize ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Adaptive Gradient Descent without Descent

Konstantin Mishchenko

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the func ...

2020

Stochastic homogenization of degenerate integral functionals with linear growth

Graph Chatbot

Chat with Graph Search

Stochastic distributed learning with gradient quantization and double-variance reduction

Proximal Point Imitation Learning

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Fluctuation estimates for the multi-cell formula in stochastic homogenization of partitions

A first-order primal-dual method with adaptivity to local smoothness

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Integral Convexity And Parabolic Systems

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

Convex Combination of Diffusion Strategies Over Networks

Adaptive Gradient Descent without Descent

Proximal Point Imitation Learning

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Stochastic distributed learning with gradient quantization and double-variance reduction

A first-order primal-dual method with adaptivity to local smoothness

Integral Convexity And Parabolic Systems

Fluctuation estimates for the multi-cell formula in stochastic homogenization of partitions

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

Adaptive Gradient Descent without Descent

Convex Combination of Diffusion Strategies Over Networks