Publications related to Augmented Lagrangian Methods for Provable and Scalable Machine Learning

, , , ,

Causal identification is at the core of the causal inference literature, where complete algorithms have been proposed to identify causal queries of interest. The validity of these algorithms hinges on the restrictive assumption of having access to a correc ...

2022

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

On the advantages of P2P ML on mobile devices

Rachid Guerraoui, Alexandre David Olivier Maurer

Many fields make use nowadays of machine learning (ML) enhanced applications for cost optimization, scheduling or forecasting, in- cluding the energy sector. However, these very ML algorithms consume a significant amount of energy, sometimes going against ...

ACM2022

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Volkan Cevher, Thomas Michaelsen Pethick

This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems. It is well-known that finding a local solution for general minimax problems is computationally intractable. This observation has recently motivat ...

2022

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Volkan Cevher, Kimon Antonakopoulos

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem’s regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this regard, existin ...

2022

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Sublinear Algorithms for Spectral Graph Clustering

Aidasadat Mousavifar

This thesis focuses on designing spectral tools for graph clustering in sublinear time. With the emergence of big data, many traditional polynomial time, and even linear time algorithms have become prohibitively expensive. Processing modern datasets requir ...

EPFL2021

Automated Design: A Journey Across Modelling, Optimization, and Education

Cyril Picard

Machine intelligence greatly impacts almost all domains of our societies. It is profoundly changing the field of mechanical engineering with new technical possibilities and processes. The education of future engineers also needs to adapt in terms of techni ...

EPFL2021

KPC: Learning-Based Model Predictive Control with Deterministic Guarantees

Colin Neil Jones, Yuning Jiang, Paul Scharnhorst, Emilio Maddalena

We propose Kernel Predictive Control (KPC), a learning-based predictive control strategy that enjoys deterministic guarantees of safety. Noise-corrupted samples of the unknown system dynamics are used to learn several models through the formalism of non-pa ...

2021

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Graph Chatbot

Chat with Graph Search

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

On the advantages of P2P ML on mobile devices

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

A first-order primal-dual method with adaptivity to local smoothness

Sublinear Algorithms for Spectral Graph Clustering

Automated Design: A Journey Across Modelling, Optimization, and Education

KPC: Learning-Based Model Predictive Control with Deterministic Guarantees

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

A first-order primal-dual method with adaptivity to local smoothness

On the advantages of P2P ML on mobile devices

KPC: Learning-Based Model Predictive Control with Deterministic Guarantees

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

UNDERGRAD: A Universal Black-Box Optimization Method with Almost Dimension-Free Convergence Rate Guarantees

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Automated Design: A Journey Across Modelling, Optimization, and Education

Sublinear Algorithms for Spectral Graph Clustering