Publications related to Stochastic optimization

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Volkan Cevher, Thomas Michaelsen Pethick

This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems. It is well-known that finding a local solution for general minimax problems is computationally intractable. This observation has recently motivat ...

2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

On the Double Descent of Random Features Models Trained with SGD

Volkan Cevher, Fanghui Liu

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/overparameterized regime. In this work, we derive precise non-asymptotic error bounds of RF regression under b ...

2022

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Negar Kiyavash, Mohammadsadegh Khorasani, Saber Salehkaleybar

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

Generalized parallel tempering on Bayesian inverse problems

Fabio Nobile, Juan Pablo Madrigal Cianci

In the current work we present two generalizations of the Parallel Tempering algorithm in the context of discrete-timeMarkov chainMonteCarlo methods for Bayesian inverse problems. These generalizations use state-dependent swapping rates, inspired by the so ...

SPRINGER2021

Optimization methods for collaborative learning

Sai Praneeth Reddy Karimireddy

A traditional machine learning pipeline involves collecting massive amounts of data centrally on a server and training models to fit the data. However, increasing concerns about the privacy and security of user's data, combined with the sheer growth in the ...

EPFL2021

Turning PathFinder Upside -Down: 1-4,xploring FPGA Switch -Blocks by Negotiating Switch Presence

Paolo Ienne, Stefan Nikolic

Automated switch-block exploration gains in importance as technology scaling brings more emphasis on the physical constraints, making it insufficient to rely on abstract measures of routability alone. In this work, we take an approach that significantly di ...

IEEE COMPUTER SOC2021

Graph-Homomorphic Perturbations For Private Decentralized Learning

Ali H. Sayed, Stefan Vlaski

Decentralized algorithms for stochastic optimization and learning rely on the diffusion of information through repeated local exchanges of intermediate estimates. Such structures are particularly appealing in situations where agents may be hesitant to shar ...

IEEE2021

A General Framework for Optimal Data-Driven Optimization

Daniel Kuhn, Tobias Sutter

We propose a statistically optimal approach to construct data-driven decisions for stochastic optimization problems. Fundamentally, a data-driven decision is simply a function that maps the available training data to a feasible action. It can always be exp ...

2021

Stochastic optimization

Graph Chatbot

Chat with Graph Search

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

On the Double Descent of Random Features Models Trained with SGD

Momentum-Based Policy Gradient with Second-Order Information

Generalized parallel tempering on Bayesian inverse problems

Optimization methods for collaborative learning

Turning PathFinder Upside -Down: 1-4,xploring FPGA Switch -Blocks by Negotiating Switch Presence

Graph-Homomorphic Perturbations For Private Decentralized Learning

A General Framework for Optimal Data-Driven Optimization

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Momentum-Based Policy Gradient with Second-Order Information

Generalized parallel tempering on Bayesian inverse problems

Optimization methods for collaborative learning

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Turning PathFinder Upside -Down: 1-4,xploring FPGA Switch -Blocks by Negotiating Switch Presence

Graph-Homomorphic Perturbations For Private Decentralized Learning

On the Double Descent of Random Features Models Trained with SGD

A General Framework for Optimal Data-Driven Optimization