Publications related to Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

2021

Computational Inverse Design of Surface-based Inflatables

Mark Pauly, Florin Isvoranu, Tian Chen

We present a computational inverse design method for a new class of surface-based inflatable structure. Our deployable structures are fabricated by fusing together two layers of inextensible sheet material along carefully selected curves. The fusing curves ...

ASSOC COMPUTING MACHINERY2021

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

Volkan Cevher, Shaul Nadav Hallak

This paper develops a methodology for regret minimization with stochastic first-order oracle feedback in online, constrained, non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, and ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

An Optimal-Storage Approach to Semidefinite Programming Using Approximate Complementarity

Volkan Cevher, Alp Yurtsever

This paper develops a new storage-optimal algorithm that provably solves almost all semidefinite programs (SDPs). This method is particularly effective for weakly constrained SDPs under appropriate regularity conditions. The key idea is to formulate an app ...

SIAM PUBLICATIONS2021

Fast Projection Onto Convex Smooth Constraints

Maryam Kamgarpour, Andreas Krause, Ilnura Usmanova

The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where ...

PMLR2021

Convergence of adaptive algorithms for constrained weakly convex optimization

Volkan Cevher, Ahmet Alacaoglu, Yurii Malitskyi

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective. We prove the

\tilde O(t^{-1/2})

rate of convergence for the squared norm of the gradient of Moreau envelope, ...

2021

Convergence without Convexity: Sampling, Optimization, and Games

Ya-Ping Hsieh

Many important problems in contemporary machine learning involve solving highly non- convex problems in sampling, optimization, or games. The absence of convexity poses significant challenges to convergence analysis of most training algorithms, and in some ...

EPFL2020

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach

Shaul Nadav Hallak

This paper introduces a method for computing points satisfying the second-order necessary optimality conditions for nonconvex minimization problems subject to a closed and convex constraint set. The method comprises two independent steps corresponding to t ...

SPRINGER/PLENUM PUBLISHERS2020

Second-Order Guarantees In Federated Learning

Ali H. Sayed, Stefan Vlaski, Elsa Rizk

Federated learning is a useful framework for centralized learning from distributed data under practical considerations of heterogeneity, asynchrony, and privacy. Federated architectures are frequently deployed in deep learning settings, which generally giv ...

IEEE2020

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Ali H. Sayed, Stefan Vlaski

Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the s ...

INT PRESS BOSTON, INC2020

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Graph Chatbot

Chat with Graph Search

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Computational Inverse Design of Surface-based Inflatables

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

An Optimal-Storage Approach to Semidefinite Programming Using Approximate Complementarity

Fast Projection Onto Convex Smooth Constraints

Convergence of adaptive algorithms for constrained weakly convex optimization

Convergence without Convexity: Sampling, Optimization, and Games

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach

Second-Order Guarantees In Federated Learning

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Regret Minimization in Stochastic Non-Convex Learning via a Proximal-Gradient Approach

An Optimal-Storage Approach to Semidefinite Programming Using Approximate Complementarity

Computational Inverse Design of Surface-based Inflatables

Fast Projection Onto Convex Smooth Constraints

Convergence of adaptive algorithms for constrained weakly convex optimization

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Finding Second-Order Stationary Points in Constrained Minimization: A Feasible Direction Approach

Convergence without Convexity: Sampling, Optimization, and Games

Second-Order Guarantees In Federated Learning

Second-order guarantees in centralized, federated and decentralized nonconvex optimization