Publication

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Related publications (166)

A Catalyst Framework for Minimax Optimization

We introduce a generic two-loop scheme for smooth minimax optimization with strongly-convex-concave objectives. Our approach applies the accelerated proximal point framework (or Catalyst) to the associated dual problem and takes full advantage of existing ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2020

A Catalyst Framework for Minimax Optimization

Negar Kiyavash

We introduce a generic \emph{two-loop} scheme for smooth minimax optimization with strongly-convex-concave objectives. Our approach applies the accelerated proximal point framework (or Catalyst) to the associated \emph{dual problem} and takes full advantag ...

2020

Conditional gradient methods for stochastically constrained convex minimization

Volkan Cevher, Ya-Ping Hsieh, Ahmet Alacaoglu, Maria-Luiza Vladarean

We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints. Instances of this template naturally arise from SDP-relaxations of combinatorial problems, whi ...

2020

Convex Combination of Diffusion Strategies Over Networks

Ali H. Sayed, Jie Chen

Combining diffusion strategies with complementary properties enables enhanced performance when they can be run simultaneously. In this article, we first propose two schemes for the convex combination of two diffusion strategies, namely, the power-normalize ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Florent Gérard Krzakala, Lenka Zdeborová

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we fo ...

Curran Associates, Inc.2020

Functional Penalised Basis Pursuit on Spheres

Matthieu Martin Jean-André Simeoni

In this work, we propose a unified theoretical and practical spherical approximation framework for functional inverse problems on the hypersphere. More specifically, we consider recovering spherical fields directly in the continuous domain using functional ...

2020

A new regret analysis for Adam-type algorithms

Volkan Cevher, Ahmet Alacaoglu

In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.). In practice, these algorithms are used with a constant first-order moment parameter 1 (typically between 0:9 and 0:99). In theory, regret guarantees for onl ...

2020

Stochastic Frank-Wolfe for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed fo ...

2019

Sampling can be faster than optimization

Nicolas Henri Bernard Flammarion

Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years. There is, however, limited theoretical understanding of the relati ...

PNAS2019

Scalable Convex Optimization Methods for Semidefinite Programming

Alp Yurtsever

With the ever-growing data sizes along with the increasing complexity of the modern problem formulations, contemporary applications in science and engineering impose heavy computational and storage burdens on the optimization algorithms. As a result, there ...

EPFL2019

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.