Publications related to Stochastic gradient descent

A Continuized View on Nesterov Acceleration

Nicolas Henri Bernard Flammarion, Raphaël Jean Berthier

We introduce the "continuized" Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter. The two variables continuously mix following a linear ordinary differential equation and take gradien ...

2021

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

Michaël Unser, Anaïs Laure Marie-Thérèse Badoual

We present a new family of active surfaces for the semiautomatic segmentation of volumetric objects in 3D biomedical images. We represent our deformable model by a subdivision surface encoded by a small set of control points and generated through a geometr ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Dan Alistarh, Fartash Faghri

As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform parallel model training. One popular communication-compression method for data-para ...

MICROTOME PUBL2021

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Differentially Private Stochastic Coordinate Descent

Rachid Guerraoui, Georgios Damaskinos

In this paper we tackle the challenge of making the stochastic coordinate descent algorithm differentially private. Compared to the classical gradient descent algorithm where updates operate on a single model vector and controlled noise addition to this ve ...

AIAA2021

Simultaneous Perturbation Stochastic Approximation of the Quantum Fisher Information

Giuseppe Carleo, Julien Sebastian Gacon, Stefan Woerner

The Quantum Fisher Information matrix (QFIM) is a central metric in promising algorithms, such as Quantum Natural Gradient Descent and Variational Quantum Imaginary Time Evolution. Computing the full QFIM for a model with d parameters, however, is computat ...

VEREIN FORDERUNG OPEN ACCESS PUBLIZIERENS QUANTENWISSENSCHAF2021

Convergence of adaptive algorithms for constrained weakly convex optimization

Volkan Cevher, Ahmet Alacaoglu, Yurii Malitskyi

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective. We prove the

\tilde O(t^{-1/2})

rate of convergence for the squared norm of the gradient of Moreau envelope, ...

2021

Robust Learning-Augmented Caching: An Experimental Study

Adam Teodor Polak

Effective caching is crucial for performance of modern-day computing systems. A key optimization problem arising in caching – which item to evict to make room for a new item – cannot be optimally solved without knowing the future. There are many classical ...

PMLR2021

Representing graphs through data with learning and optimal transport

Hermina Petric Maretic

Graphs offer a simple yet meaningful representation of relationships between data. Thisrepresentation is often used in machine learning algorithms in order to incorporate structuralor geometric information about data. However, it can also be used in an inv ...

EPFL2021

Neural network surrogate of QuaLiKiz using JET experimental data to populate training space

Henri Weisen, Yann Camenen

Within integrated tokamak plasma modeling, turbulent transport codes are typically the computational bottleneck limiting their routine use outside of post-discharge analysis. Neural network (NN) surrogates have been used to accelerate these calculations wh ...

AMER INST PHYSICS2021

Stochastic gradient descent

Graph Chatbot

Chat with Graph Search

A Continuized View on Nesterov Acceleration

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Differentially Private Stochastic Coordinate Descent

Simultaneous Perturbation Stochastic Approximation of the Quantum Fisher Information

Convergence of adaptive algorithms for constrained weakly convex optimization

Robust Learning-Augmented Caching: An Experimental Study

Representing graphs through data with learning and optimal transport

Neural network surrogate of QuaLiKiz using JET experimental data to populate training space

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

A Continuized View on Nesterov Acceleration

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

Differentially Private Stochastic Coordinate Descent

Simultaneous Perturbation Stochastic Approximation of the Quantum Fisher Information

Representing graphs through data with learning and optimal transport

Convergence of adaptive algorithms for constrained weakly convex optimization

Robust Learning-Augmented Caching: An Experimental Study

Neural network surrogate of QuaLiKiz using JET experimental data to populate training space