Publications related to STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

ADAGRAD Avoids Saddle Points

Kimon Antonakopoulos, Xiao Wang

Adaptive first-order methods in optimization are prominent in machine learning and data science owing to their ability to automatically adapt to the landscape of the function being optimized. However, their convergence guarantees are typically stated in te ...

2022

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Negar Kiyavash, Mohammadsadegh Khorasani, Saber Salehkaleybar

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

Physics-enhanced machine learning with symmetry-adapted and long-range representations

Andrea Grisafi

Theoretical and computational approaches to the study of materials and molecules have, over the last few decades, progressed at an exponential rate. Yet, the possibility of producing numerical predictions that are on par with experimental measurements is t ...

EPFL2021

Computational Inverse Design of Surface-based Inflatables

Mark Pauly, Florin Isvoranu, Tian Chen

We present a computational inverse design method for a new class of surface-based inflatable structure. Our deployable structures are fabricated by fusing together two layers of inextensible sheet material along carefully selected curves. The fusing curves ...

ASSOC COMPUTING MACHINERY2021

Large Steps in Inverse Rendering of Geometry

Wenzel Alban Jakob, Baptiste Alexandre Marie Philippe Claude Nicolet

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to mi ...

ASSOC COMPUTING MACHINERY2021

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Ali H. Sayed, Stefan Vlaski

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available wor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation

François Fleuret, Camilla Carta

We present DepthInSpace, a self-supervised deep-learning method for depth estimation using a structured-light camera. The design of this method is motivated by the commercial use case of embedded depth sensors in nowadays smartphones. We first propose to u ...

IEEE2021

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In Part I [3] of this work we established that agents cluster around a network centroid and pr ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Second-Order Guarantees In Federated Learning

Ali H. Sayed, Stefan Vlaski, Elsa Rizk

Federated learning is a useful framework for centralized learning from distributed data under practical considerations of heterogeneity, asynchrony, and privacy. Federated architectures are frequently deployed in deep learning settings, which generally giv ...

IEEE2020

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Graph Chatbot

Chat with Graph Search

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

ADAGRAD Avoids Saddle Points

Momentum-Based Policy Gradient with Second-Order Information

Physics-enhanced machine learning with symmetry-adapted and long-range representations

Computational Inverse Design of Surface-based Inflatables

Large Steps in Inverse Rendering of Geometry

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Second-Order Guarantees In Federated Learning

Physics-enhanced machine learning with symmetry-adapted and long-range representations

Large Steps in Inverse Rendering of Geometry

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation

Computational Inverse Design of Surface-based Inflatables

Second-Order Guarantees In Federated Learning

ADAGRAD Avoids Saddle Points

Momentum-Based Policy Gradient with Second-Order Information

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points