Publication

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Related publications (32)

On the choice of metric in gradient-based theories of brain function

Wulfram Gerstner, Johanni Michael Brea, Jean-Pascal Théodor Pfister

The idea that the brain functions so as to minimize certain costs pervades theoretical neuroscience. Because a cost function by itself does not predict how the brain finds its minima, additional assumptions about the optimization method need to be made to ...

2020

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Updates

Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stocha ...

MICROTOME PUBL2020

3D nanometrology of transparent objects by phase calibration of a basic bright-field microscope for multiple illumination apertures

Martinus Gijs, Daniel Migliozzi, Bingying Zhao

Optical retrieval of the structure of transparent objects at the nanoscale requires adapted techniques capable of probing their interaction with light. Here, we considered a method based on calibration of the defocusing with partially coherent illumination ...

2020

Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint

Pascal Frossard, Mireille El Gheche, Giovanni Chierchia

In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its application to larg ...

IEEE2019

MATHICSE Technical Report : Analysis of stochastic gradient methods for PDE-constrained optimal control problems with uncertain parameters

Fabio Nobile, Sebastian Krumscheid, Matthieu Claude Martin

We consider the numerical approximation of a risk-averse optimal control problem for an elliptic partial differential equation (PDE) with random coefficients. Specifically, the control function is a deterministic, dis- tributed forcing term that minimizes ...

MATHICSE2018

On the linear convergence of the stochastic gradient method with constant step-size

Volkan Cevher, Cong Bang Vu

The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size γ (SGM-CS). In this paper, we provide a necessary condition, for the linear convergence of SGM-CS, t ...

2018

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Volkan Cevher, Junhong Lin

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes o ...

2018

Accelerated ADMM based on Accelerated Douglas-Rachford Splitting

Colin Neil Jones, Ivan Pejcic

Alternating direction method of multipliers (ADMM) is a form of augmented Lagrangian optimisation algorithm that found its place in many new applications in recent years. This paper explores a possibility for an upgrade of the ADMM by extrapolation-based a ...

Ieee2016

Stochastic Spectral Descent for Restricted Boltzmann Machines.

Volkan Cevher

Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated with sampling methods. However, the gradient estimation is a ...

2015

Spline based iterative phase retrieval algorithm for X-ray differential phase contrast radiography

Michaël Unser, Masih Nilchian, Marco Stampanoni

Differential phase contrast imaging using grating interferometer is a promising alternative to conventional X-ray radiographic methods. It provides the absorption, differential phase and scattering information of the underlying sample simultaneously. Phase ...

Optical Soc Amer2015

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.