Publications related to Universal and adaptive methods for robust stochastic optimization

Scalable constrained optimization

Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...

EPFL2024

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Anastasiia Koloskova

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Multi-agent Learning with Privacy Guarantees

Elsa Rizk

A multi-agent system consists of a collection of decision-making or learning agents subjected to streaming observations from some real-world phenomenon. The goal of the system is to solve some global learning or optimization problem in a distributed or dec ...

EPFL2023

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

Emilio Maddalena

This thesis is situated at the crossroads between machine learning and control engineering. Our contributions are both theoretical, through proposing a new uncertainty quantification methodology in a kernelized context; and experimental, through investigat ...

EPFL2023

Stochastic adaptive resampling for the estimation of discrete choice models

Michel Bierlaire, Nicola Marco Ortelli, Matthieu Marie Cochon de Lapparent

In the field of choice modeling, the availability of ever-larger datasets has the potential to significantly expand our understanding of human behavior, but this prospect is limited by the poor scalability of discrete choice models (DCMs): as sample sizes ...

2023

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Giancarlo Ferrari Trecate, Maryam Kamgarpour, Yuning Jiang, Baiwei Guo

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optim ...

2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Nirupam Gupta, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Universal and adaptive methods for robust stochastic optimization

Graph Chatbot

Chat with Graph Search

Scalable constrained optimization

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

On the Generalization of Stochastic Gradient Descent with Momentum

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Multi-agent Learning with Privacy Guarantees

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

Stochastic adaptive resampling for the estimation of discrete choice models

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

Robust Collaborative Learning with Linear Gradient Overhead

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Scalable constrained optimization

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Multi-agent Learning with Privacy Guarantees

Robust Collaborative Learning with Linear Gradient Overhead

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

Stochastic adaptive resampling for the estimation of discrete choice models

Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

On the Generalization of Stochastic Gradient Descent with Momentum