Publications related to Variational Information Maximization in Gaussian Channels

Mutual Information Disentangles Interactions from Changing Environments

Daniel Maria Busiello, Giorgio Nicoletti

Real-world systems are characterized by complex interactions of their internal degrees of freedom, while living in ever-changing environments whose net effect is to act as additional couplings. Here, we introduce a paradigmatic interacting model in a switc ...

AMER PHYSICAL SOC2021

Oblivious Sketching of High-Degree Polynomial Kernels

Mikhail Kapralov, Amir Zandieh

Kernel methods are fundamental tools in machine learning that allow detection of non-linear dependencies between data without explicitly constructing feature vectors in high dimensional spaces. A major disadvantage of kernel methods is their poor scalabili ...

ASSOC COMPUTING MACHINERY2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Strengthened Information-theoretic Bounds on the Generalization Error

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The following problem is considered: given a joint distribution P XY and an event E, bound P XY (E) in terms of P X P Y (E) (where P X P Y is the product of the marginals of P XY ) and a measure of dependence of X and Y. Such bounds have direct application ...

IEEE2019

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Nicolas Macris, Jean François Emmanuel Barbier, Chun Lam Chan

We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping the model to a ran ...

IEEE2019

Subadditivity Beyond Trees and the Chi-Squared Mutual Information

Emmanuel Abbé, Enric Boix Adsera

Evans et al. [1] proved the subadditivity of the mutual information in the broadcasting on tree model with binary vertex labels and symmetric edge channels. They raised the question of whether such subadditivity extends to loopy graphs in some appropriate ...

IEEE2019

Mutual Information for Low-Rank Even-Order Symmetric Tensor Factorization

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its mutual information when the tensor is of even order. The proof uses the adaptive interpolation method, for which rank-on ...

IEEE2019

High-Dimensional Inference on Dense Graphs with Applications to Coding Theory and Machine Learning

Mohamad Baker Dia

We are living in the era of "Big Data", an era characterized by a voluminous amount of available data. Such amount is mainly due to the continuing advances in the computational capabilities for capturing, storing, transmitting and processing data. However, ...

EPFL2018

Neural Tanget Kernel: Convergence and Generalization in Neural Networks

At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit [12, 9], thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Variational Information Maximization in Gaussian Channels

Graph Chatbot

Chat with Graph Search

Mutual Information Disentangles Interactions from Changing Environments

Oblivious Sketching of High-Degree Polynomial Kernels

Robust Generalization via $\alpha$-Mutual Information

Robust Generalization via f−Mutual Information

Strengthened Information-theoretic Bounds on the Generalization Error

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Subadditivity Beyond Trees and the Chi-Squared Mutual Information

Mutual Information for Low-Rank Even-Order Symmetric Tensor Factorization

High-Dimensional Inference on Dense Graphs with Applications to Coding Theory and Machine Learning

Neural Tanget Kernel: Convergence and Generalization in Neural Networks

Mutual Information Disentangles Interactions from Changing Environments

Oblivious Sketching of High-Degree Polynomial Kernels

Robust Generalization via $\alpha$-Mutual Information

Strengthened Information-theoretic Bounds on the Generalization Error

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Robust Generalization via f−Mutual Information

Subadditivity Beyond Trees and the Chi-Squared Mutual Information

Mutual Information for Low-Rank Even-Order Symmetric Tensor Factorization

High-Dimensional Inference on Dense Graphs with Applications to Coding Theory and Machine Learning

Neural Tanget Kernel: Convergence and Generalization in Neural Networks