Publications related to A proximal Newton framework for composite minimization: Graph learning without Cholesky decompositions and matrix inversions

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Giancarlo Ferrari Trecate, Maryam Kamgarpour, Yuning Jiang, Baiwei Guo

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

A Multiconfigurational Wave Function Implementation of the Frenkel Exciton Model for Molecular Aggregates

Francesco Aquilante

We present an implementation of the Frenkel exciton model into the OpenMolcas program package enabling calculations of collective electronic excited states of molecular aggregates based on a multiconfigurational wave function description of the individual ...

AMER CHEMICAL SOC2023

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Volkan Cevher, Kimon Antonakopoulos

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem’s regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this regard, existin ...

2022

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Nicolas Henri Bernard Flammarion, Aditya Vardhan Varre, Loucas Pillaud-Vivien

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs ...

2021

A proximal Newton framework for composite minimization: Graph learning without Cholesky decompositions and matrix inversions

Graph Chatbot

Chat with Graph Search

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Universal and adaptive methods for robust stochastic optimization

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

A Multiconfigurational Wave Function Implementation of the Frenkel Exciton Model for Molecular Aggregates

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

A first-order primal-dual method with adaptivity to local smoothness

Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A Multiconfigurational Wave Function Implementation of the Frenkel Exciton Model for Molecular Aggregates

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Universal and adaptive methods for robust stochastic optimization

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

A first-order primal-dual method with adaptivity to local smoothness

Last iterate convergence of SGD for Least-Squares in the Interpolation regime