Publications related to Composite convex minimization involving self-concordant-like cost functions

Universal and adaptive methods for robust stochastic optimization

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Mathematical Foundations of Robust and Distributionally Robust Optimization

Daniel Kuhn, Jianzhe Zhen, Wolfram Wiesemann

Robust and distributionally robust optimization are modeling paradigms for decision-making under uncertainty where the uncertain parameters are only known to reside in an uncertainty set or are governed by any probability distribution from within an ambigu ...

2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

2021

Adaptive Gradient Descent without Descent

Konstantin Mishchenko

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the func ...

2020

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach

Shaul Nadav Hallak

This paper introduces a method for computing points satisfying the second-order necessary optimality conditions for nonconvex minimization problems subject to a closed and convex constraint set. The method comprises two independent steps corresponding to t ...

SPRINGER/PLENUM PUBLISHERS2020

Stochastic Frank-Wolfe for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed fo ...

2019

Composite convex minimization involving self-concordant-like cost functions

Graph Chatbot

Chat with Graph Search

Universal and adaptive methods for robust stochastic optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A first-order primal-dual method with adaptivity to local smoothness

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Mathematical Foundations of Robust and Distributionally Robust Optimization

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Adaptive Gradient Descent without Descent

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach

Stochastic Frank-Wolfe for Composite Convex Minimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Universal and adaptive methods for robust stochastic optimization

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Mathematical Foundations of Robust and Distributionally Robust Optimization

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A first-order primal-dual method with adaptivity to local smoothness

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Adaptive Gradient Descent without Descent

Stochastic Frank-Wolfe for Composite Convex Minimization

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach