Publications related to Stochastic Three-Composite Convex Minimization

Universal and adaptive methods for robust stochastic optimization

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Computational Inverse Design of Surface-based Inflatables

Mark Pauly, Florin Isvoranu, Tian Chen

We present a computational inverse design method for a new class of surface-based inflatable structure. Our deployable structures are fabricated by fusing together two layers of inextensible sheet material along carefully selected curves. The fusing curves ...

ASSOC COMPUTING MACHINERY2021

A Newton Frank-Wolfe method for constrained self-concordant minimization

Volkan Cevher, Quoc Tran Dinh

We develop a new Newton Frank-Wolfe algorithm to solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO). Unlike L-smooth convex functions, where the Lipschitz continuity of the objective gradient holds gl ...

SPRINGER2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

2021

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Volkan Cevher, Shaul Nadav Hallak

This paper develops a methodology for regret minimization with stochastic first-order oracle feedback in online, constrained, non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, and ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Stochastic Three-Composite Convex Minimization

Graph Chatbot

Chat with Graph Search

Universal and adaptive methods for robust stochastic optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A first-order primal-dual method with adaptivity to local smoothness

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Computational Inverse Design of Surface-based Inflatables

A Newton Frank-Wolfe method for constrained self-concordant minimization

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Universal and adaptive methods for robust stochastic optimization

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

A first-order primal-dual method with adaptivity to local smoothness

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Computational Inverse Design of Surface-based Inflatables

A Newton Frank-Wolfe method for constrained self-concordant minimization