Publications related to Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

A Functional Perspective on Information Measures

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

From Generalisation Error to Transportation-cost Inequalities and Back

Michael Christoph Gastpar, Amedeo Roberto Esposito

In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities. Exposing the underlying pattern behind both approaches we are able to generalise them and go beyond Kullback- Leibler Divergences/Mutu ...

2022

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Locally Differentially-Private Randomized Response for Discrete Distribution Learning

Michael Christoph Gastpar, Adriano Pastore

We consider a setup in which confidential i.i.d. samples X1, . . . , Xn from an unknown finite-support distribution p are passed through n copies of a discrete privatization chan- nel (a.k.a. mechanism) producing outputs Y1, . . . , Yn. The channel law gua ...

2021

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Nicolas Henri Bernard Flammarion, Xiang Cheng

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an under-damped form of the Langevin algorithm perfor ...

INT STATISTICAL INST2021

A Comparison of optimal measurement-system design with engineering judgement for bridge load testing

Ian Smith, Numa Joy Bertola

Due to conservative approaches in construction design and practice, infrastructure often has hidden reserve capacity. When quantified, this reserve has potential to improve decisions related to asset management. Field measurements, collected through load t ...

2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Florent Gérard Krzakala, Jean François Emmanuel Barbier, Mohamad Baker Dia

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Entropy as a tool for crystal discovery

Pablo Miguel Piaggi

The computational prediction of crystal structures has emerged as an useful alternative to expensive and often cumbersome experiments. We propose an approach to the prediction of crystal structures and polymorphism based on reproducing the crystallization ...

EPFL2019

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Graph Chatbot

Chat with Graph Search

A Functional Perspective on Information Measures

From Generalisation Error to Transportation-cost Inequalities and Back

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Locally Differentially-Private Randomized Response for Discrete Distribution Learning

Is there an analog of Nesterov acceleration for gradient-based MCMC?

A Comparison of optimal measurement-system design with engineering judgement for bridge load testing

Robust Generalization via f−Mutual Information

Robust Generalization via $\alpha$-Mutual Information

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Entropy as a tool for crystal discovery

A Functional Perspective on Information Measures

Locally Differentially-Private Randomized Response for Discrete Distribution Learning

A Comparison of optimal measurement-system design with engineering judgement for bridge load testing

From Generalisation Error to Transportation-cost Inequalities and Back

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Entropy as a tool for crystal discovery

Robust Generalization via $\alpha$-Mutual Information

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Robust Generalization via f−Mutual Information