Publications associées à Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Data Downloaded via Parachute from a NASA Super-Pressure Balloon

David Richard Harvey, Mathilde Jauzac, Richard Massey, Lun Li

In April 2023, the superBIT telescope was lifted to the Earth's stratosphere by a helium-filled super-pressure balloon to acquire astronomical imaging from above (99.5% of) the Earth's atmosphere. It was launched from New Zealand and then, for 40 days, cir ...

Basel2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Nirupam Gupta, Shuo Liu

This paper considers the Byzantine fault-tolerance problem in distributed stochastic gradient descent (D-SGD) method - a popular algorithm for distributed multi-agent machine learning. In this problem, each agent samples data points independently from a ce ...

IEEE COMPUTER SOC2021

Variance-Reduced Stochastic Learning Under Random Reshuffling

Ali H. Sayed, Bicheng Ying, Kun Yuan

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizer. The existing convergence results assume uniform d ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Multi-Gradient Descent for Multi-Objective Recommender Systems

Boi Faltings, Claudiu-Cristian Musat, Diego Matteo Antognini

Recommender systems need to mirror the complexity of the environment they are applied in. The more we know about what might benefit the user, the more objectives the recommender system has. In addition there may be multiple stakeholders - sellers, buyers, ...

2020

Multi-armed Bandits in Action

Farnood Salehi

Making decisions is part and parcel of being human. Among a set of actions, we want to choose the one that has the highest reward. But the uncertainty of the outcome prevents us from always making the right decision. Making decisions under uncertainty can ...

EPFL2020

Scalable Stochastic Optimization: Scenario Reduction with Guarantees

Kilian Schindler

Stochastic optimization is a popular modeling paradigm for decision-making under uncertainty and has a wide spectrum of applications in management science, economics and engineering. However, the stochastic optimization models one faces in practice are int ...

EPFL2020

Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

Graph Chatbot

Chattez avec Graph Search

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Data Downloaded via Parachute from a NASA Super-Pressure Balloon

Universal and adaptive methods for robust stochastic optimization

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Variance-Reduced Stochastic Learning Under Random Reshuffling

Multi-Gradient Descent for Multi-Objective Recommender Systems

Multi-armed Bandits in Action

Scalable Stochastic Optimization: Scenario Reduction with Guarantees

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Multi-Gradient Descent for Multi-Objective Recommender Systems

Multi-armed Bandits in Action

Universal and adaptive methods for robust stochastic optimization

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Variance-Reduced Stochastic Learning Under Random Reshuffling

Data Downloaded via Parachute from a NASA Super-Pressure Balloon

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Scalable Stochastic Optimization: Scenario Reduction with Guarantees

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination