Publication

Exact Diffusion for Distributed Optimization and Learning---Part II: Convergence Analysis

Related publications (52)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Optimization Methods for Control: From Embedded Programmable Hardware to Data-Driven Process Optimization

Harsh Ambarishkumar Shukla

The research community has been making significant progress in hardware implementation, numerical computing and algorithm development for optimization-based control. However, there are two key challenges that still have to be overcome for optimization-base ...

EPFL2021

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

, ,

This paper presents an adaptive combination strategy for distributed learning over diffusion networks. Since learning relies on the collaborative processing of the stochastic information at the dispersed agents, the overall performance can be improved by d ...

IEEE2021

Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization

Ali H. Sayed, Kun Yuan

This paper introduces a new algorithm for consensus optimization in a multi-agent network, where all agents collaboratively find a minimizer for the sum of their private functions. All decentralized algorithms rely on communications between adjacent nodes. ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Volkan Cevher, Ali Kavis, Shaul Nadav Hallak

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm’s convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with prob ...

2020

Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Ali H. Sayed, Stefan Vlaski

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to no ...

IEEE2020

, ,

Part I of this paper developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to the larger set of locally balanced left-sto ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Learning Generalized Nash Equilibria in a Class of Convex Games

Maryam Kamgarpour

We consider multiagent decision making where each agent optimizes its convex cost function subject to individual and coupling constraints. The constraint sets are compact convex subsets of a Euclidean space. To learn Nash equilibria, we propose a novel dis ...

2019

Exact Diffusion for Distributed Optimization and Learning-Part I: Algorithm Development

Ali H. Sayed, Bicheng Ying, Kun Yuan

This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range an ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019