Publications related to Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

On design strategies for binary dielectric metasurfaces based on the Fourier modal method

Binary dielectric metasurfaces are arrays of sub-wavelength structures that act as a thin layer of artificial material. They are generally lossless and relatively simple to fabricate since only a single structuring step is required. By carefully designing ...

EPFL2020

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Volkan Cevher, Ahmet Alacaoglu, Armin Eftekhari, Fabian Ricardo Latorre Gomez, Mehmet Fatih Sahin

We propose a practical inexact augmented Lagrangian method (iALM) for nonconvex problems with nonlinear constraints. We characterize the total computational complexity of our method subject to a verifiable geometric condition, which is closely related to t ...

2019

Improved Bounds for Discretization of Langevin Diffusions: Near-Optimal Rates without Convexity

Nicolas Henri Bernard Flammarion

We consider minimizing a nonconvex, smooth function

f

on a Riemannian manifold

\mathcal{M}

. We show that a perturbed version of Riemannian gradient descent algorithm converges to a second-order stationary point (and hence is able to escape saddle point ...

2019

Escaping from saddle points on Riemannian manifolds

Nicolas Henri Bernard Flammarion, Yue Sun

We consider minimizing a nonconvex, smooth function f on a Riemannian manifold M. We show that a perturbed version of Riemannian gradient descent algorithm converges to a second-order stationary point (and hence is able to escape saddle points on the manif ...

2019

Polynomial Escape-Time From Saddle Points In Distributed Non-Convex Optimization

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In this work we establish that agents cluster around a network centroid in the mean-fourth sen ...

IEEE2019

Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint

Pascal Frossard, Mireille El Gheche, Giovanni Chierchia

In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its application to larg ...

IEEE2019

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Ali H. Sayed, Bicheng Ying

The analysis in Part I [1] revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization. These algorithms are used when the risk functions are non-smooth or involve non-differentiable components. They have b ...

Elsevier2018

Exact Diffusion for Distributed Optimization and Learning---Part II: Convergence Analysis

Ali H. Sayed, Bicheng Ying, Kun Yuan

Part I of this work developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to a larger set of combination policies than ea ...

2017

Asynchronous updates for stochastic gradient descent

Alberto Silvio Chiappa

Finding convergence rates for numerical optimization algorithms is an important task, because it gives a justification to their use in solving practical problems, while also providing a way to compare their efficiency. This is especially useful in an async ...

2017

Stochastic Three-Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Cong Bang Vu

We propose a stochastic optimization method for the minimization of the sum of three convex functions, one of which has Lipschitz continuous gradient as well as restricted strong convexity. Our approach is most suitable in the setting where it is computati ...

2016

Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Graph Chatbot

Chat with Graph Search

On design strategies for binary dielectric metasurfaces based on the Fourier modal method

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Improved Bounds for Discretization of Langevin Diffusions: Near-Optimal Rates without Convexity

Escaping from saddle points on Riemannian manifolds

Polynomial Escape-Time From Saddle Points In Distributed Non-Convex Optimization

Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Exact Diffusion for Distributed Optimization and Learning---Part II: Convergence Analysis

Asynchronous updates for stochastic gradient descent

Stochastic Three-Composite Convex Minimization

Asynchronous updates for stochastic gradient descent

Exact Diffusion for Distributed Optimization and Learning---Part II: Convergence Analysis

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

On design strategies for binary dielectric metasurfaces based on the Fourier modal method

Stochastic Three-Composite Convex Minimization

Escaping from saddle points on Riemannian manifolds

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Improved Bounds for Discretization of Langevin Diffusions: Near-Optimal Rates without Convexity

Polynomial Escape-Time From Saddle Points In Distributed Non-Convex Optimization

Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint