Publications related to A Non-Euclidean Gradient Descent Framework for Non-Convex Matrix Factorization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Giancarlo Ferrari Trecate, Maryam Kamgarpour, Yuning Jiang, Baiwei Guo

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

2023

A Spatial Branch and Bound Algorithm for Continuous Pricing with Advanced Discrete Choice Demand Modeling

Michel Bierlaire

In this paper, we present a spatial branch and bound algorithm to tackle the continuous pricing problem, where demand is captured by an advanced discrete choice model (DCM). Advanced DCMs, like mixed logit or latent class models, are capable of modeling de ...

2023

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Adaptive Gradient Descent without Descent

Konstantin Mishchenko

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the func ...

2020

Implications of ANEC for SCFTs in four dimensions

Andrea Manenti, Alessandro Vichi

We explore consequences of the Averaged Null Energy Condition (ANEC) for scaling dimensions Delta of operators in four-dimensional N = 1 superconformal field theories. We show that in many cases the ANEC bounds are stronger than the corresponding unitarity ...

SPRINGER2020

Scalable Semidefinite Programming

Volkan Cevher, Alp Yurtsever

Semidefinite programming (SDP) is a powerful framework from convex optimization that has striking potential for data science applications. This paper develops a provably correct algorithm for solving large SDP problems by economizing on both the storage an ...

2020

A Non-Euclidean Gradient Descent Framework for Non-Convex Matrix Factorization

Graph Chatbot

Chat with Graph Search

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Universal and adaptive methods for robust stochastic optimization

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

A Spatial Branch and Bound Algorithm for Continuous Pricing with Advanced Discrete Choice Demand Modeling

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A first-order primal-dual method with adaptivity to local smoothness

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Adaptive Gradient Descent without Descent

Implications of ANEC for SCFTs in four dimensions

Scalable Semidefinite Programming

Universal and adaptive methods for robust stochastic optimization

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

A Spatial Branch and Bound Algorithm for Continuous Pricing with Advanced Discrete Choice Demand Modeling

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

A first-order primal-dual method with adaptivity to local smoothness

Implications of ANEC for SCFTs in four dimensions

Adaptive Gradient Descent without Descent

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Scalable Semidefinite Programming