Publications related to Universal and adaptive methods for robust stochastic optimization

Enriching the Computational Toolbox for Organocatalysis

Organocatalysis has evolved significantly over the last decades, becoming a pillar of synthetic chemistry, but traditional theoretical approaches based on quantum mechanical computations to investigate reaction mechanisms and provide rationalizations of ca ...

EPFL2023

Bayesian Optimization for Chemical Reactions

Philippe Schwaller, Jeff Guo, Bojana Rankovic

Reaction optimization is challenging and traditionally delegated to domain experts who iteratively pro-pose increasingly optimal experiments. Problematically, the reaction landscape is complex and often requires hundreds of experiments to reach convergence ...

Bern2023

The statistical complexity of early-stopped mirror descent

Tomas Vaskevicius, Varun Kanade

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford2023

Memory of Motion for Initializing Optimization in Robotics

Teguh Santoso Lembono

Many robotics problems are formulated as optimization problems. However, most optimization solvers in robotics are locally optimal and the performance depends a lot on the initial guess. For challenging problems, the solver will often get stuck at poor loc ...

EPFL2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Extra Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

Volkan Cevher, Kimon Antonakopoulos, Ali Kavis

This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions. Our algorithm achieves O(σ/T‾‾√) convergence when the oracle feedback is stochastic with variance σ2, and improves its convergence to O(1/ ...

2022

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Negar Kiyavash, Mohammadsadegh Khorasani, Saber Salehkaleybar

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

SIAM PUBLICATIONS2022

Universal and adaptive methods for robust stochastic optimization

Graph Chatbot

Chat with Graph Search

Enriching the Computational Toolbox for Organocatalysis

Bayesian Optimization for Chemical Reactions

The statistical complexity of early-stopped mirror descent

Memory of Motion for Initializing Optimization in Robotics

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Extra Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Momentum-Based Policy Gradient with Second-Order Information

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Memory of Motion for Initializing Optimization in Robotics

The statistical complexity of early-stopped mirror descent

Bayesian Optimization for Chemical Reactions

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Predicting in Uncertain Environments: Methods for Robust Machine Learning

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Extra Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

Enriching the Computational Toolbox for Organocatalysis

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Momentum-Based Policy Gradient with Second-Order Information