Publications related to Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

On the Generalization of Stochastic Gradient Descent with Momentum

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

The statistical complexity of early-stopped mirror descent

Tomas Vaskevicius, Varun Kanade

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford2023

Alternation makes the adversary weaker in two-player games

Volkan Cevher, Efstratios Panteleimon Skoulakis, Luca Viano, Ali Kavis

Motivated by alternating game-play in two-player games, we study an altenating variant of the Online Linear Optimization (OLO). In alternating OLO, a learner at each round t ∈[n] selects a vector xt and then an adversary selects a cost-vector ct ∈[−1,1]n. ...

2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

No-regret learning in games with noisy feedback: Faster rates and adaptivity via learning rate separation

Volkan Cevher, Kimon Antonakopoulos, Ya-Ping Hsieh

We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully ...

2022

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Graph Chatbot

Chat with Graph Search

On the Generalization of Stochastic Gradient Descent with Momentum

Universal and adaptive methods for robust stochastic optimization

Deep Learning Generalization with Limited and Noisy Labels

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

The statistical complexity of early-stopped mirror descent

Alternation makes the adversary weaker in two-player games

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

No-regret learning in games with noisy feedback: Faster rates and adaptivity via learning rate separation

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

On the Generalization of Stochastic Gradient Descent with Momentum

Universal and adaptive methods for robust stochastic optimization

Deep Learning Generalization with Limited and Noisy Labels

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Alternation makes the adversary weaker in two-player games

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Predicting in Uncertain Environments: Methods for Robust Machine Learning

No-regret learning in games with noisy feedback: Faster rates and adaptivity via learning rate separation

The statistical complexity of early-stopped mirror descent