Publications related to Chaining Mutual Information and Tightening Generalization Bounds

A Functional Perspective on Information Measures

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Michael Christoph Gastpar, Erixhen Sula

This paper presents explicit solutions for two related non-convex information extremization problems due to Gray and Wyner in the Gaussian case. The first problem is the Gray-Wyner network subject to a sum-rate constraint on the two private links. Here, ou ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Visual Focus of Attention Estimation in 3D Scene with an Arbitrary Number of Targets

Jean-Marc Odobez, Rémy Alain Siegfried

Visual Focus of Attention (VFOA) estimation in conversation is challenging as it relies on difficult to estimate information (gaze) combined with scene features like target positions and other contextual information (speaking status) allowing to disambigua ...

IEEE COMPUTER SOC2021

Deep Learning Building on Prior Ischemic Core Segmentation Improves Prediction of Infarction After Stroke

Dimitri Nestor Alice Van De Ville, Elisabeth Anne Dirren, Guillaume Paul Olivier Leclerc

Introduction: Imaging studies are used to guide patient selection for acute stroke treatment. Perfusion CT (pCT) is widely used to identify the acute ischemic core and penumbra, but the prediction of the final infarct remains challenging. With the advent o ...

LIPPINCOTT WILLIAMS & WILKINS2021

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Florent Gérard Krzakala, Jean François Emmanuel Barbier, Mohamad Baker Dia

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Strengthened Information-theoretic Bounds on the Generalization Error

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The following problem is considered: given a joint distribution P XY and an event E, bound P XY (E) in terms of P X P Y (E) (where P X P Y is the product of the marginals of P XY ) and a measure of dependence of X and Y. Such bounds have direct application ...

IEEE2019

Learning and Adaptive Data Analysis via Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

There has been growing interest in studying connections between generalization error of learning algorithms and information measures. In this work, we generalize a result that employs the maximal leakage, a measure of leakage of information, and explore ho ...

IEEE2019

Chaining Mutual Information and Tightening Generalization Bounds

Graph Chatbot

Chat with Graph Search

A Functional Perspective on Information Measures

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Visual Focus of Attention Estimation in 3D Scene with an Arbitrary Number of Targets

Deep Learning Building on Prior Ischemic Core Segmentation Improves Prediction of Infarction After Stroke

Robust Generalization via $\alpha$-Mutual Information

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Robust Generalization via f−Mutual Information

Strengthened Information-theoretic Bounds on the Generalization Error

Learning and Adaptive Data Analysis via Maximal Leakage

A Functional Perspective on Information Measures

Visual Focus of Attention Estimation in 3D Scene with an Arbitrary Number of Targets

Deep Learning Building on Prior Ischemic Core Segmentation Improves Prediction of Infarction After Stroke

Learning and Adaptive Data Analysis via Maximal Leakage

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Robust Generalization via $\alpha$-Mutual Information

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Strengthened Information-theoretic Bounds on the Generalization Error

Robust Generalization via f−Mutual Information