Publications related to Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

Ultrafast ultrasound imaging, characterized by high frame rates, generates low-quality images. Convolutional neural networks (CNNs) have demonstrated great potential to enhance image quality without compromising the frame rate. However, CNNs have been most ...

2023

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Michael Christoph Gastpar, Adrien Vandenbroucque, Amedeo Roberto Esposito

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of f-Divergences. The results are then applied to specific settings of interest and compared to other notable results i ...

2022

Statistical Emulation of Neural Simulators: Application to Neocortical L2/3 Large Basket Cells

Werner Alfons Hilda Van Geit, Oren Amsalem, Idan Segev

Many scientific systems are studied using computer codes that simulate the phenomena of interest. Computer simulation enables scientists to study a broad range of possible conditions, generating large quantities of data at a faster rate than the laboratory ...

FRONTIERS MEDIA SA2022

From Generalisation Error to Transportation-cost Inequalities and Back

Michael Christoph Gastpar, Amedeo Roberto Esposito

In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities. Exposing the underlying pattern behind both approaches we are able to generalise them and go beyond Kullback- Leibler Divergences/Mutu ...

2022

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Nicolas Henri Bernard Flammarion, Xiang Cheng

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an under-damped form of the Langevin algorithm perfor ...

INT STATISTICAL INST2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks.

Konstantinos Pitas

Deep neural networks have been empirically successful in a variety of tasks, however their theoretical understanding is still poor. In particular, modern deep neural networks have many more parameters than training data. Thus, in principle they should over ...

EPFL2020

Fast Mean Estimation with Sub-Gaussian Rates

Nicolas Henri Bernard Flammarion

We present an improved analysis of the Euler-Maruyama discretization of the Langevin diffusion. Our analysis does not require global contractivity, and yields polynomial dependence on the time horizon. Compared to existing approaches, we make an additional ...

2019

Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models

Graph Chatbot

Chat with Graph Search

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Statistical Emulation of Neural Simulators: Application to Neocortical L2/3 Large Basket Cells

From Generalisation Error to Transportation-cost Inequalities and Back

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Robust Generalization via f−Mutual Information

Robust Generalization via $\alpha$-Mutual Information

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks.

Fast Mean Estimation with Sub-Gaussian Rates

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Statistical Emulation of Neural Simulators: Application to Neocortical L2/3 Large Basket Cells

From Generalisation Error to Transportation-cost Inequalities and Back

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Robust Generalization via f−Mutual Information

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks.

Fast Mean Estimation with Sub-Gaussian Rates

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Robust Generalization via $\alpha$-Mutual Information