Publications related to On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Safe non-smooth black-box optimization with application to policy search

Maryam Kamgarpour, Andreas Krause, Ilnura Usmanova

For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth ...

PMLR2020

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Daniel Kuhn, Viet Anh Nguyen, Soroosh Shafieezadeh Abadeh, Wolfram Wiesemann

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood ...

2019

Statistical Aspects of Wasserstein Distances

Victor Panaretos, Yoav Zemel

Wasserstein distances are metrics on probability distributions inspired by the problem of optimal mass transportation. Roughly speaking, they measure the minimal effort required to reconfigure the probability mass of one distribution in order to recover th ...

ANNUAL REVIEWS2019

Exact Diffusion for Distributed Optimization and Learning-Part II: Convergence Analysis

Ali H. Sayed, Bicheng Ying, Kun Yuan

Part I of this paper developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to the larger set of locally balanced left-sto ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Efficient learning of smooth probability functions from Bernoulli tests with guarantees.

Volkan Cevher, Paul Thierry Yves Rolland, Ali Kavis

We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests. We provide a scalable algorithm for efficiently solving this problem with rigorous guarantees. In particular, we prove the convergence rate ...

2019

Avoidance of Convex and Concave Obstacles with Convergence ensured through Contraction

Aude Billard, Lukas Huber

This paper presents a closed-form approach to obstacle avoidance for multiple moving convex and star-shaped concave obstacles. The method takes inspiration in harmonic-potential fields. It inherits the convergence properties of harmonic potentials. We prov ...

2019

Dynamic Average Diffusion With Randomized Coordinate Updates

Ali H. Sayed, Bicheng Ying, Kun Yuan

This work derives and analyzes an online learning strategy for tracking the average of time-varying distributed signals by relying on randomized coordinate-descent updates. During each iteration, each agent selects or observes a random entry of the observa ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Ali H. Sayed, Bicheng Ying

The analysis in Part I [1] revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization. These algorithms are used when the risk functions are non-smooth or involve non-differentiable components. They have b ...

Elsevier2018

Convergence analysis of Padé approximations for Helmholtz frequency response problems

Fabio Nobile, Francesca Bonizzoni

The present work concerns the approximation of the solution map

S

associated to the parametric Helmholtz boundary value problem, i.e., the map which associates to each (real) wavenumber belonging to a given interval of interest the corresponding solution ...

2018

Non-local Functionals Related to the Total Variation and Connections with Image Processing

Hoài-Minh Nguyên

We present new results concerning the approximation of the total variation,

\int_{\Omega} |\nabla u|

, of a function

u

by non-local, non-convex functionals of the form $$ \Lambda_\delta u = \int_{\Omega} \int_{\Omega} \frac{\delta \varphi \big( |u(x) - ...

2018

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Graph Chatbot

Chat with Graph Search

Safe non-smooth black-box optimization with application to policy search

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Statistical Aspects of Wasserstein Distances

Exact Diffusion for Distributed Optimization and Learning-Part II: Convergence Analysis

Efficient learning of smooth probability functions from Bernoulli tests with guarantees.

Avoidance of Convex and Concave Obstacles with Convergence ensured through Contraction

Dynamic Average Diffusion With Randomized Coordinate Updates

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Convergence analysis of Padé approximations for Helmholtz frequency response problems

Non-local Functionals Related to the Total Variation and Connections with Image Processing

Safe non-smooth black-box optimization with application to policy search

Efficient learning of smooth probability functions from Bernoulli tests with guarantees.

Avoidance of Convex and Concave Obstacles with Convergence ensured through Contraction

Dynamic Average Diffusion With Randomized Coordinate Updates

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Statistical Aspects of Wasserstein Distances

Exact Diffusion for Distributed Optimization and Learning-Part II: Convergence Analysis

Convergence analysis of Padé approximations for Helmholtz frequency response problems

Non-local Functionals Related to the Total Variation and Connections with Image Processing