Concept

Variation of information

Related publications (74)

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Florent Gérard Krzakala, Jean François Emmanuel Barbier, Mohamad Baker Dia

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Framework to Approximate the Value of Information of Bridge Load Testing for Reserve Capacity Assessment

Ian Smith, Numa Joy Bertola, Marco Proverbio

In developed countries, structural assessment of existing bridges should not be performed using the same conservative models that are used at the design stage. Field measurements of real behavior provide additional information for the inference of previous ...

2020

Strengthened Information-theoretic Bounds on the Generalization Error

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The following problem is considered: given a joint distribution P XY and an event E, bound P XY (E) in terms of P X P Y (E) (where P X P Y is the product of the marginals of P XY ) and a measure of dependence of X and Y. Such bounds have direct application ...

IEEE2019

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Nicolas Macris, Jean François Emmanuel Barbier, Chun Lam Chan

We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping the model to a ran ...

IEEE2019

Theoretical foundations of forward feature selection methods based on mutual information

Francisco Santos Paredes Quartin de Macedo

Feature selection problems arise in a variety of applications, such as microarray analysis, clinical prediction, text categorization, image classification and face recognition, multi-label learning, and classification of internet traffic. Among the various ...

ELSEVIER SCIENCE BV2019

Learning and Adaptive Data Analysis via Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

There has been growing interest in studying connections between generalization error of learning algorithms and information measures. In this work, we generalize a result that employs the maximal leakage, a measure of leakage of information, and explore ho ...

IEEE2019

Measuring transient phase-amplitude coupling using local mutual information

Here we demonstrate the suitability of a local mutual information measure for estimating the temporal dynamics of cross-frequency coupling (CFC) in brain electrophysiological signals. In CFC, concurrent activity streams in different frequency ranges intera ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2019

Mutual Information for Low-Rank Even-Order Symmetric Tensor Factorization

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its mutual information when the tensor is of even order. The proof uses the adaptive interpolation method, for which rank-on ...

IEEE2019

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020